[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:26 dist-test-slave-zj8q krb5kdc[11218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:26 dist-test-slave-zj8q krb5kdc[11218](info): set up 2 sockets
Aug 10 06:43:26 dist-test-slave-zj8q krb5kdc[11218](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 06:43:27.946801 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.372s user 0.005s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:43:28 dist-test-slave-zj8q krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:43:28.141983 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:36363
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:36363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:28.461063 11237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:28.461352 11237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:28.461505 11237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:28.468560 11237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:28.468664 11237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:28.468719 11237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:28.468775 11237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:28.468818 11237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:28.478302 11237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:36363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:36363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:28.479475 11237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:28.480757 11237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:28.492224 11253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:28.492269 11248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:28.492177 11249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:28.493103 11237 server_base.cc:1034] running on GCE node
I20260810 06:43:28.493851 11237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:28.495148 11237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:28.496344 11237 hybrid_clock.cc:648] HybridClock initialized: now 1786344208496298 us; error 97 us; skew 500 ppm
I20260810 06:43:28.496685 11237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:28 dist-test-slave-zj8q krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344208, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:28.499853 11237 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:43:28.501259 11237 webserver.cc:460] Webserver started at http://127.10.242.62:33397/ using document root <none> and password file <none>
I20260810 06:43:28.501892 11237 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:28.502000 11237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:28.502341 11237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:28.505286 11237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403"
format_stamp: "Formatted at 2026-08-10 06:43:28 on dist-test-slave-zj8q"
server_key: "f8a10c23ba6208284c76775e615b3aa9"
server_key_iv: "8cfc0a1109ed0161ce783c0da833b6e5"
server_key_version: "encryptionkey@0"
I20260810 06:43:28.506016 11237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403"
format_stamp: "Formatted at 2026-08-10 06:43:28 on dist-test-slave-zj8q"
server_key: "f8a10c23ba6208284c76775e615b3aa9"
server_key_iv: "8cfc0a1109ed0161ce783c0da833b6e5"
server_key_version: "encryptionkey@0"
I20260810 06:43:28.511111 11237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:43:28.514406 11265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:28.515600 11237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:43:28.515769 11237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403"
format_stamp: "Formatted at 2026-08-10 06:43:28 on dist-test-slave-zj8q"
server_key: "f8a10c23ba6208284c76775e615b3aa9"
server_key_iv: "8cfc0a1109ed0161ce783c0da833b6e5"
server_key_version: "encryptionkey@0"
I20260810 06:43:28.515940 11237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:28.534701 11237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:28.538836 11237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:28.539099 11237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:28.556398 11237 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:36363
I20260810 06:43:28.556420 11345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:36363 every 8 connection(s)
I20260810 06:43:28.557741 11237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:28.561306 11346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:28.567175 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 11237
I20260810 06:43:28.567517 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:43:28.570027 11208 external_mini_cluster.cc:1428] Setting key d28b260990482202665c5d744b711083
I20260810 06:43:28.573537 11346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403: Bootstrap starting.
I20260810 06:43:28.576853 11346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:28.577850 11346 log.cc:824] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:28.580410 11346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403: No bootstrap required, opened a new log
I20260810 06:43:28.588932 11346 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36363 } }
I20260810 06:43:28.589404 11346 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:28.589521 11346 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdfb1f7e0c944fbc91ae9603bb7c1403, State: Initialized, Role: FOLLOWER
I20260810 06:43:28.590097 11346 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36363 } }
I20260810 06:43:28.590353 11346 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:28.590488 11346 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:28.590643 11346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:28.592934 11346 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36363 } }
I20260810 06:43:28.593513 11346 leader_election.cc:302] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdfb1f7e0c944fbc91ae9603bb7c1403; no voters:
I20260810 06:43:28.594235 11346 leader_election.cc:288] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:28.594304 11351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:28.595542 11351 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [term 1 LEADER]: Becoming Leader. State: Replica: cdfb1f7e0c944fbc91ae9603bb7c1403, State: Running, Role: LEADER
I20260810 06:43:28.596064 11351 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36363 } }
I20260810 06:43:28.596429 11346 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:28.598665 11353 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdfb1f7e0c944fbc91ae9603bb7c1403. Latest consensus state: current_term: 1 leader_uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36363 } } }
I20260810 06:43:28.599035 11353 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:28.599591 11352 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfb1f7e0c944fbc91ae9603bb7c1403" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36363 } } }
I20260810 06:43:28.600631 11352 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:28.601044 11357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 06:43:28 dist-test-slave-zj8q krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:28.610320 11357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:28.619071 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:28.574831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38392 (local address 127.10.242.62:36363)
0810 06:43:28.575764 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:28.575779 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:28.577060 (+ 1281us) server_negotiation.cc:407] Connection header received
0810 06:43:28.588301 (+ 11241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:28.588330 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:28.588819 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:28.589243 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:28.592373 (+ 3130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:28.594501 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:28.597015 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:28.598171 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:28.599951 (+ 1780us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:28.606616 (+ 6665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:28.606628 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:28.606650 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:28.606700 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:28.611365 (+ 4665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:28.612423 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:28.612430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:28.612439 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:28.612650 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:28.613254 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:28.613259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:28.613265 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:28.614117 (+ 852us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:28.616716 (+ 2599us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:28.616994 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 06:43:28.617425 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":422,"threads_started":1}
I20260810 06:43:28.628715 11357 catalog_manager.cc:1347] Generated new cluster ID: 5ffed11d231d45458ad3b76969183a7a
I20260810 06:43:28.628821 11357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:28.663133 11357 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:28.664454 11357 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:28.676394 11357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdfb1f7e0c944fbc91ae9603bb7c1403: Generated new TSK 0
I20260810 06:43:28.677202 11357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:43:28.723430 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36363
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:29.045770 11381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:29.046038 11381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:29.046205 11381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:29.053232 11381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:29.053339 11381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:29.053493 11381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:43:29.063054 11381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36363
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:29.064337 11381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:29.065671 11381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:29.078133 11391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:29.078225 11389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:29.078253 11388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:29.079416 11381 server_base.cc:1034] running on GCE node
I20260810 06:43:29.079950 11381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:29.080777 11381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:29.081965 11381 hybrid_clock.cc:648] HybridClock initialized: now 1786344209081911 us; error 46 us; skew 500 ppm
I20260810 06:43:29.082304 11381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:29 dist-test-slave-zj8q krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344209, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:29.086052 11381 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:43:29.087596 11381 webserver.cc:460] Webserver started at http://127.10.242.1:42529/ using document root <none> and password file <none>
I20260810 06:43:29.088291 11381 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:29.088394 11381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:29.088655 11381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:29.091526 11381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "3660a2738440470e8c2456dcf6377312"
format_stamp: "Formatted at 2026-08-10 06:43:29 on dist-test-slave-zj8q"
server_key: "68c1261b52522b02a69f7f601d74d444"
server_key_iv: "e4c9f40c3ef9b4218a1094d63c2cd464"
server_key_version: "encryptionkey@0"
I20260810 06:43:29.092238 11381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "3660a2738440470e8c2456dcf6377312"
format_stamp: "Formatted at 2026-08-10 06:43:29 on dist-test-slave-zj8q"
server_key: "68c1261b52522b02a69f7f601d74d444"
server_key_iv: "e4c9f40c3ef9b4218a1094d63c2cd464"
server_key_version: "encryptionkey@0"
I20260810 06:43:29.097221 11381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:43:29.100562 11401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:29.101982 11381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:43:29.102128 11381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "3660a2738440470e8c2456dcf6377312"
format_stamp: "Formatted at 2026-08-10 06:43:29 on dist-test-slave-zj8q"
server_key: "68c1261b52522b02a69f7f601d74d444"
server_key_iv: "e4c9f40c3ef9b4218a1094d63c2cd464"
server_key_version: "encryptionkey@0"
I20260810 06:43:29.102324 11381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:29.135576 11381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:29.140066 11381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:29.140336 11381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:29.141362 11381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:29.142786 11381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:29.142860 11381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:29.142966 11381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:29.143009 11381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:29.173689 11381 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:40825
I20260810 06:43:29.173760 11579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:40825 every 8 connection(s)
I20260810 06:43:29.175010 11381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:43:29.179080 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 11381
I20260810 06:43:29.179327 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:43:29.180133 11208 external_mini_cluster.cc:1428] Setting key 42eb0c31787801288cb5554a375efe6e
Aug 10 06:43:29 dist-test-slave-zj8q krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344209, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:29.197268 11591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:29.177641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:46065 (local address 127.10.242.62:36363)
0810 06:43:29.178091 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:29.178095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:29.180834 (+ 2739us) server_negotiation.cc:407] Connection header received
0810 06:43:29.181659 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:29.181665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:29.181753 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:29.181849 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:29.183856 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:29.184612 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:29.186748 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:29.187178 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:29.187278 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:29.191081 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:29.191087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:29.191091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:29.191129 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:29.194220 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:29.194913 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:29.194919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:29.194922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:29.195067 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:29.195451 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:29.195456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:29.195460 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:29.195797 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:29.195964 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:29.196668 (+ 704us) server_negotiation.cc:299] Negotiation successful
0810 06:43:29.196896 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":234,"threads_started":1}
I20260810 06:43:29.198006 11592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:29.178306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:36363 (local address 127.10.242.1:46065)
0810 06:43:29.179092 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:29.179150 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:29.181408 (+ 2258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:29.182108 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:29.182116 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:29.182867 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:29.183665 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:29.183690 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:29.184763 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:29.184768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:29.186607 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:29.186622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:29.187430 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:29.187437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:29.187701 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:29.188377 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:29.188411 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:29.190836 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:29.194341 (+ 3505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:29.194348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:29.194361 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:29.194779 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:29.195161 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:29.195164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:29.195166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:29.195341 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:29.195903 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:29.195908 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:29.196210 (+ 302us) client_negotiation.cc:769] Sending connection context
0810 06:43:29.196504 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 06:43:29.196915 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":269,"threads_started":1}
I20260810 06:43:29.199465 11581 heartbeater.cc:347] Connected to a master server at 127.10.242.62:36363
I20260810 06:43:29.199750 11581 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
I20260810 06:43:29.200459 11581 heartbeater.cc:511] Master 127.10.242.62:36363 requested a full tablet report, sending...
I20260810 06:43:29.202653 11289 ts_manager.cc:194] Registered new tserver with Master: 3660a2738440470e8c2456dcf6377312 (127.10.242.1:40825)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
I20260810 06:43:29.204229 11289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:46065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:43:29.223013 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36363
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:29.541641 11599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:29.541900 11599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:29.541998 11599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:29.549129 11599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:29.549232 11599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:29.549355 11599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:43:29.558753 11599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36363
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:29.559880 11599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:29.561126 11599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:29.574447 11604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:29.574520 11605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:29.574483 11611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:29.575177 11599 server_base.cc:1034] running on GCE node
I20260810 06:43:29.575726 11599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:29.576560 11599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:29.577766 11599 hybrid_clock.cc:648] HybridClock initialized: now 1786344209577709 us; error 47 us; skew 500 ppm
I20260810 06:43:29.578052 11599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:29 dist-test-slave-zj8q krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344209, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:29.581045 11599 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:43:29.582541 11599 webserver.cc:460] Webserver started at http://127.10.242.2:45289/ using document root <none> and password file <none>
I20260810 06:43:29.583382 11599 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:29.583520 11599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:29.583884 11599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:29.587896 11599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "be24fdc5e0b048e48fb6a11382aeaecc"
format_stamp: "Formatted at 2026-08-10 06:43:29 on dist-test-slave-zj8q"
server_key: "c8aa6567d2c91112217846f66265d490"
server_key_iv: "bdc5eb8acd31ed1bb3094fc484ac6381"
server_key_version: "encryptionkey@0"
I20260810 06:43:29.588855 11599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "be24fdc5e0b048e48fb6a11382aeaecc"
format_stamp: "Formatted at 2026-08-10 06:43:29 on dist-test-slave-zj8q"
server_key: "c8aa6567d2c91112217846f66265d490"
server_key_iv: "bdc5eb8acd31ed1bb3094fc484ac6381"
server_key_version: "encryptionkey@0"
I20260810 06:43:29.595342 11599 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 06:43:29.599668 11622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:29.600986 11599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:43:29.601137 11599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "be24fdc5e0b048e48fb6a11382aeaecc"
format_stamp: "Formatted at 2026-08-10 06:43:29 on dist-test-slave-zj8q"
server_key: "c8aa6567d2c91112217846f66265d490"
server_key_iv: "bdc5eb8acd31ed1bb3094fc484ac6381"
server_key_version: "encryptionkey@0"
I20260810 06:43:29.601310 11599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:29.629597 11599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:29.633929 11599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:29.634249 11599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:29.635293 11599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:29.636610 11599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:29.636686 11599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:29.636803 11599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:29.636852 11599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:29.669313 11599 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:33469
I20260810 06:43:29.669416 11805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:33469 every 8 connection(s)
I20260810 06:43:29.670662 11599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:43:29.678885 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 11599
I20260810 06:43:29.679071 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:43:29.679483 11208 external_mini_cluster.cc:1428] Setting key e2804f4df8e33b380b526cdc484ffeba
Aug 10 06:43:29 dist-test-slave-zj8q krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344209, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:29.691557 11591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:29.673702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:46449 (local address 127.10.242.62:36363)
0810 06:43:29.673906 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:29.673913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:29.675194 (+ 1281us) server_negotiation.cc:407] Connection header received
0810 06:43:29.677075 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:29.677082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:29.677170 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:29.677300 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:29.678828 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:29.679916 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:29.681915 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:29.682381 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:29.682495 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:29.685712 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:29.685717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:29.685720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:29.685752 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:29.688225 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:29.689041 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:29.689046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:29.689050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:29.689230 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:29.689754 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:29.689760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:29.689763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:29.690111 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:29.690296 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:29.691058 (+ 762us) server_negotiation.cc:299] Negotiation successful
0810 06:43:29.691302 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 06:43:29.692682 11811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:29.674181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:36363 (local address 127.10.242.2:46449)
0810 06:43:29.675019 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:29.675064 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:29.676813 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:29.677472 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:29.677479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:29.677969 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:29.678634 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:29.678662 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:29.680091 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:29.680097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:29.681779 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:29.681792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:29.682595 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:29.682603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:29.682819 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:29.683411 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:29.683433 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:29.685491 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:29.688344 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:29.688353 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:29.688367 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:29.688904 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:29.689375 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:29.689379 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:29.689382 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:29.689614 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:29.690265 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:29.690272 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:29.690648 (+ 376us) client_negotiation.cc:769] Sending connection context
0810 06:43:29.690931 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 06:43:29.691392 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":357,"threads_started":1}
I20260810 06:43:29.694036 11807 heartbeater.cc:347] Connected to a master server at 127.10.242.62:36363
I20260810 06:43:29.694337 11807 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:29.694994 11807 heartbeater.cc:511] Master 127.10.242.62:36363 requested a full tablet report, sending...
I20260810 06:43:29.696390 11289 ts_manager.cc:194] Registered new tserver with Master: be24fdc5e0b048e48fb6a11382aeaecc (127.10.242.2:33469)
I20260810 06:43:29.697173 11289 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:46449
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:43:29.721184 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36363
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:32951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:30.039417 11815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:30.039717 11815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:30.039875 11815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:30.047042 11815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:30.047152 11815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:30.047303 11815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:43:30.056926 11815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:32951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36363
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:30.058136 11815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:30.059522 11815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:30.072021 11827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:30.072085 11824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:30.072113 11823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:30.072801 11815 server_base.cc:1034] running on GCE node
I20260810 06:43:30.073364 11815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:30.074277 11815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:30.075438 11815 hybrid_clock.cc:648] HybridClock initialized: now 1786344210075378 us; error 50 us; skew 500 ppm
I20260810 06:43:30.075747 11815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344210, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:30.078651 11815 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:43:30.079988 11815 webserver.cc:460] Webserver started at http://127.10.242.3:40797/ using document root <none> and password file <none>
I20260810 06:43:30.080605 11815 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:30.080730 11815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:30.081028 11815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:30.083978 11815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "ff3f4352f11942b189de6a59ec22d8e3"
format_stamp: "Formatted at 2026-08-10 06:43:30 on dist-test-slave-zj8q"
server_key: "df814f5a811784fd725cf92abf3b36a0"
server_key_iv: "1edca760a8456e99d443fa15028dec99"
server_key_version: "encryptionkey@0"
I20260810 06:43:30.084769 11815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "ff3f4352f11942b189de6a59ec22d8e3"
format_stamp: "Formatted at 2026-08-10 06:43:30 on dist-test-slave-zj8q"
server_key: "df814f5a811784fd725cf92abf3b36a0"
server_key_iv: "1edca760a8456e99d443fa15028dec99"
server_key_version: "encryptionkey@0"
I20260810 06:43:30.090543 11815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 06:43:30.093895 11836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:30.095193 11815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:30.095353 11815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "ff3f4352f11942b189de6a59ec22d8e3"
format_stamp: "Formatted at 2026-08-10 06:43:30 on dist-test-slave-zj8q"
server_key: "df814f5a811784fd725cf92abf3b36a0"
server_key_iv: "1edca760a8456e99d443fa15028dec99"
server_key_version: "encryptionkey@0"
I20260810 06:43:30.095583 11815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:30.117623 11815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:30.121592 11815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:30.121862 11815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:30.122869 11815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:30.124194 11815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:30.124281 11815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:30.124403 11815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:30.124457 11815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:30.155313 11815 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:38733
I20260810 06:43:30.155364 12021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:38733 every 8 connection(s)
I20260810 06:43:30.156672 11815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:43:30.164539 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 11815
I20260810 06:43:30.164719 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:43:30.165271 11208 external_mini_cluster.cc:1428] Setting key f5ab6570ab3daed75876d30095111c8a
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344210, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:30.176615 11591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:30.159239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:58035 (local address 127.10.242.62:36363)
0810 06:43:30.159407 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:30.159412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:30.160577 (+ 1165us) server_negotiation.cc:407] Connection header received
0810 06:43:30.162389 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:30.162394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:30.162462 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:30.162552 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:30.164011 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.164825 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.166842 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.167455 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.167571 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:30.170942 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:30.170947 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:30.170950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:30.170981 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:30.173626 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.174359 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.174365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.174369 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.174534 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.174896 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.174900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.174903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.175251 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:30.175395 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:30.176044 (+ 649us) server_negotiation.cc:299] Negotiation successful
0810 06:43:30.176276 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:43:30.177335 12028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:30.159709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:36363 (local address 127.10.242.3:58035)
0810 06:43:30.160403 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:30.160447 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:30.162065 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:30.162736 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:30.162743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:30.163256 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:30.163849 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:30.163867 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.164990 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.164996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:30.166706 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:30.166719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.167686 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.167693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:30.167888 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:30.168493 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:30.168527 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:30.170758 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:30.173771 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:30.173777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:30.173787 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:30.174237 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:30.174630 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:30.174633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:30.174635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:30.174801 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:30.175351 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:30.175356 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:30.175697 (+ 341us) client_negotiation.cc:769] Sending connection context
0810 06:43:30.175906 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 06:43:30.176265 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":266,"threads_started":1}
I20260810 06:43:30.178731 12024 heartbeater.cc:347] Connected to a master server at 127.10.242.62:36363
I20260810 06:43:30.179041 12024 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:30.179883 12024 heartbeater.cc:511] Master 127.10.242.62:36363 requested a full tablet report, sending...
I20260810 06:43:30.181397 11288 ts_manager.cc:194] Registered new tserver with Master: ff3f4352f11942b189de6a59ec22d8e3 (127.10.242.3:38733)
I20260810 06:43:30.182284 11288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:58035
I20260810 06:43:30.193181 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:43:30.207211 11581 heartbeater.cc:503] Master 127.10.242.62:36363 was elected leader, sending a full tablet report...
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344210, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344210, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:43:30.239925 12033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:30.225654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52190 (local address 127.10.242.1:40825)
0810 06:43:30.226404 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:30.226411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:30.226439 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:43:30.226500 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:30.226507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:30.226723 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:30.226849 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:30.228189 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.229209 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.230446 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.231008 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.231102 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:30.234806 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:30.234820 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:30.234837 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:30.234870 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:30.237606 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.238188 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.238198 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.238202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.238327 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.238660 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.238663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.238665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.238994 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:30.239238 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:30.239443 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 06:43:30.239638 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":422,"threads_started":1}
W20260810 06:43:30.241218 11453 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52190
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[11218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:43:30.286209 12033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:30.273440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52192 (local address 127.10.242.1:40825)
0810 06:43:30.273666 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:30.273673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:30.273817 (+ 144us) server_negotiation.cc:407] Connection header received
0810 06:43:30.273967 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:30.273971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:30.274080 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:30.274202 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:30.275499 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.276233 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.277415 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.277875 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.277973 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:30.281574 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:30.281579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:30.281582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:30.281613 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:30.284073 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.284645 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.284649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.284651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.284769 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.285041 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.285044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.285046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.285285 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:30.285456 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:30.285720 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 06:43:30.285879 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 06:43:30.287633 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 11381
I20260810 06:43:30.299583 12033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:30.288501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52198 (local address 127.10.242.1:40825)
0810 06:43:30.288664 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:30.288669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:30.288949 (+ 280us) server_negotiation.cc:407] Connection header received
0810 06:43:30.289093 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:30.289098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:30.289162 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:30.289234 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:30.290390 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.291047 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.292280 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.292820 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.292946 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:30.295075 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:30.295080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:30.295083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:30.295111 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:30.297297 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.297880 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.297886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.297889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.298044 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.298419 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.298425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.298428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.298679 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:30.298892 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:30.299141 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 06:43:30.299286 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:43:30.300217 11572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:30.380730 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 11381
I20260810 06:43:30.405884 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 11599
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:43:30.420720 12041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:30.406777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38794 (local address 127.10.242.2:33469)
0810 06:43:30.407252 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:30.407259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:30.407285 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:43:30.407346 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:30.407352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:30.407549 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:30.407695 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:30.408792 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.409785 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.410994 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.411538 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.411634 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:30.415380 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:30.415391 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:30.415405 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:30.415444 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:30.418208 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.418827 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.418834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.418837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.418963 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.419345 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.419348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.419350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.419731 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:30.419963 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:30.420171 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 06:43:30.420422 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":239,"threads_started":1}
I20260810 06:43:30.421994 11800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:30.503114 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 11599
I20260810 06:43:30.524799 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 11815
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[11218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:43:30.539892 12047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:30.525773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45202 (local address 127.10.242.3:38733)
0810 06:43:30.526432 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:30.526440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:30.526471 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:43:30.526532 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:30.526538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:30.526749 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:30.526874 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:30.528070 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.529080 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.530411 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:30.530966 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:30.531061 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:30.534902 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:30.534914 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:30.534928 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:30.534962 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:30.537595 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.538129 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.538135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.538138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.538304 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:30.538598 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:30.538600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:30.538602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:30.538915 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:30.539144 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:30.539342 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 06:43:30.539535 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":331,"threads_started":1}
I20260810 06:43:30.541109 12011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:30.622104 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 11815
I20260810 06:43:30.640656 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 11237
I20260810 06:43:30.641865 11341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:30.743661 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 11237
2026-08-10T06:43:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4228 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[12055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[12055](info): set up 2 sockets
Aug 10 06:43:30 dist-test-slave-zj8q krb5kdc[12055](info): commencing operation
krb5kdc: starting...
W20260810 06:43:31.935283 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:43:32 dist-test-slave-zj8q krb5kdc[12055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:43:32.122648 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:43401
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:43401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:32.438102 12074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:32.438400 12074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:32.438539 12074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:32.445451 12074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:32.445581 12074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:32.445658 12074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:32.445739 12074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:32.445801 12074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:32.455138 12074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:43401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:43401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:32.456319 12074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:32.457547 12074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:32.468549 12082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:32.468588 12085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:32.468626 12087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:32.469583 12074 server_base.cc:1034] running on GCE node
I20260810 06:43:32.470358 12074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:32.471683 12074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:32.472890 12074 hybrid_clock.cc:648] HybridClock initialized: now 1786344212472837 us; error 40 us; skew 500 ppm
I20260810 06:43:32.473223 12074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:32 dist-test-slave-zj8q krb5kdc[12055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344212, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:32.479904 12074 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:43:32.481254 12074 webserver.cc:460] Webserver started at http://127.10.242.62:42307/ using document root <none> and password file <none>
I20260810 06:43:32.481880 12074 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:32.481997 12074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:32.482347 12074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:32.485153 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "7667c30698864b4cb652d1502c7e3765"
format_stamp: "Formatted at 2026-08-10 06:43:32 on dist-test-slave-zj8q"
server_key: "f76a7e05c4e0900eb3004c4d3073f985"
server_key_iv: "bbf11971e401bbefcc1b587598432fe3"
server_key_version: "encryptionkey@0"
I20260810 06:43:32.485884 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "7667c30698864b4cb652d1502c7e3765"
format_stamp: "Formatted at 2026-08-10 06:43:32 on dist-test-slave-zj8q"
server_key: "f76a7e05c4e0900eb3004c4d3073f985"
server_key_iv: "bbf11971e401bbefcc1b587598432fe3"
server_key_version: "encryptionkey@0"
I20260810 06:43:32.490622 12074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260810 06:43:32.493783 12098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:32.494978 12074 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:43:32.495143 12074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "7667c30698864b4cb652d1502c7e3765"
format_stamp: "Formatted at 2026-08-10 06:43:32 on dist-test-slave-zj8q"
server_key: "f76a7e05c4e0900eb3004c4d3073f985"
server_key_iv: "bbf11971e401bbefcc1b587598432fe3"
server_key_version: "encryptionkey@0"
I20260810 06:43:32.495318 12074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:32.527519 12074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:32.531262 12074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:32.531517 12074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:32.547227 12074 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:43401
I20260810 06:43:32.547235 12188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:43401 every 8 connection(s)
I20260810 06:43:32.548450 12074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:32.549292 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 12074
I20260810 06:43:32.549521 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:43:32.550290 11208 external_mini_cluster.cc:1428] Setting key dd40542feecaba24992a66671a59d3af
I20260810 06:43:32.552628 12192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:43:32 dist-test-slave-zj8q krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:32.564961 12192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765: Bootstrap starting.
I20260810 06:43:32.568909 12192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:32.570083 12192 log.cc:824] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:32.573318 12192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765: No bootstrap required, opened a new log
I20260810 06:43:32.573477 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:32.552099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.10.242.62:43401)
0810 06:43:32.552958 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:32.552969 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:32.553009 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:43:32.554665 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:32.554687 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:32.555052 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:32.555362 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:32.557141 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:32.558535 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:32.560184 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:32.560820 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:32.560955 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:32.565449 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:32.565459 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:32.565477 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:32.565533 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:32.569269 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:32.569894 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:32.569903 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:32.569911 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:32.570116 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:32.570477 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:32.570482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:32.570485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:32.571091 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:32.571308 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:32.571834 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 06:43:32.572230 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":373,"threads_started":1}
I20260810 06:43:32.581137 12192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7667c30698864b4cb652d1502c7e3765" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43401 } }
I20260810 06:43:32.581455 12192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:32.581525 12192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7667c30698864b4cb652d1502c7e3765, State: Initialized, Role: FOLLOWER
I20260810 06:43:32.582079 12192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7667c30698864b4cb652d1502c7e3765" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43401 } }
I20260810 06:43:32.582374 12192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:32.582528 12192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:32.582713 12192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:32.584728 12192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7667c30698864b4cb652d1502c7e3765" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43401 } }
I20260810 06:43:32.585171 12192 leader_election.cc:302] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7667c30698864b4cb652d1502c7e3765; no voters:
I20260810 06:43:32.585697 12192 leader_election.cc:288] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:32.585809 12198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:32.586931 12198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [term 1 LEADER]: Becoming Leader. State: Replica: 7667c30698864b4cb652d1502c7e3765, State: Running, Role: LEADER
I20260810 06:43:32.587411 12198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7667c30698864b4cb652d1502c7e3765" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43401 } }
I20260810 06:43:32.587707 12192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:32.590188 12200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7667c30698864b4cb652d1502c7e3765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7667c30698864b4cb652d1502c7e3765" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43401 } } }
I20260810 06:43:32.590278 12201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7667c30698864b4cb652d1502c7e3765. Latest consensus state: current_term: 1 leader_uuid: "7667c30698864b4cb652d1502c7e3765" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7667c30698864b4cb652d1502c7e3765" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43401 } } }
I20260810 06:43:32.590572 12200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:32.590601 12201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:32.592504 12207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:32.597721 12207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:32.607218 12207 catalog_manager.cc:1347] Generated new cluster ID: 67c97bb8540c46cea9dbe65815f93f61
I20260810 06:43:32.607321 12207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:32.625351 12207 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:32.626590 12207 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:32.640793 12207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7667c30698864b4cb652d1502c7e3765: Generated new TSK 0
I20260810 06:43:32.641552 12207 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:43:32.693297 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:43401
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:33.009111 12234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:33.009384 12234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:33.009537 12234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:33.016623 12234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:33.016724 12234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:33.016841 12234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:43:33.026805 12234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:43401
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:33.028012 12234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:33.029295 12234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:33.041991 12243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:33.042024 12247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:33.041955 12244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:33.042737 12234 server_base.cc:1034] running on GCE node
I20260810 06:43:33.043298 12234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:33.044173 12234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:33.045374 12234 hybrid_clock.cc:648] HybridClock initialized: now 1786344213045319 us; error 46 us; skew 500 ppm
I20260810 06:43:33.045680 12234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:33 dist-test-slave-zj8q krb5kdc[12055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344213, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:33.048661 12234 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:43:33.049962 12234 webserver.cc:460] Webserver started at http://127.10.242.1:40347/ using document root <none> and password file <none>
I20260810 06:43:33.050642 12234 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:33.050748 12234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:33.051011 12234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:33.053766 12234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "12fe090a834145d3b9161c4b86db16c3"
format_stamp: "Formatted at 2026-08-10 06:43:33 on dist-test-slave-zj8q"
server_key: "bbf97807e1615363f773bae9211f96e4"
server_key_iv: "bd292a8e1d19fea454ffbecf7214b14a"
server_key_version: "encryptionkey@0"
I20260810 06:43:33.054538 12234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "12fe090a834145d3b9161c4b86db16c3"
format_stamp: "Formatted at 2026-08-10 06:43:33 on dist-test-slave-zj8q"
server_key: "bbf97807e1615363f773bae9211f96e4"
server_key_iv: "bd292a8e1d19fea454ffbecf7214b14a"
server_key_version: "encryptionkey@0"
I20260810 06:43:33.059269 12234 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 06:43:33.062443 12261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:33.063619 12234 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:43:33.063760 12234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "12fe090a834145d3b9161c4b86db16c3"
format_stamp: "Formatted at 2026-08-10 06:43:33 on dist-test-slave-zj8q"
server_key: "bbf97807e1615363f773bae9211f96e4"
server_key_iv: "bd292a8e1d19fea454ffbecf7214b14a"
server_key_version: "encryptionkey@0"
I20260810 06:43:33.063913 12234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:33.078764 12234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:33.082414 12234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:33.082657 12234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:33.083513 12234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:33.084724 12234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:33.084795 12234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:33.084882 12234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:33.084924 12234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:33.119817 12234 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:41563
I20260810 06:43:33.119900 12449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:41563 every 8 connection(s)
I20260810 06:43:33.121276 12234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:43:33.131712 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 12234
I20260810 06:43:33.131915 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:43:33.132522 11208 external_mini_cluster.cc:1428] Setting key 91d3522dcb4b7949dd5990c30b35bcce
Aug 10 06:43:33 dist-test-slave-zj8q krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344213, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:33.141620 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:33.123711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:43229 (local address 127.10.242.62:43401)
0810 06:43:33.124133 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:33.124138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:33.125120 (+ 982us) server_negotiation.cc:407] Connection header received
0810 06:43:33.127012 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:33.127017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:33.127085 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:33.127196 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:33.128669 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:33.129364 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:33.130939 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:33.131348 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:33.131463 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:33.135258 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:33.135265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:33.135270 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:33.135308 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:33.138546 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:33.139264 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:33.139270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:33.139273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:33.139424 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:33.139823 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:33.139827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:33.139830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:33.140196 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:33.140328 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:33.141049 (+ 721us) server_negotiation.cc:299] Negotiation successful
0810 06:43:33.141274 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":196,"threads_started":1}
I20260810 06:43:33.142675 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:33.124266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:43401 (local address 127.10.242.1:43229)
0810 06:43:33.124956 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:33.124993 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:33.126736 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:33.127444 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:33.127452 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:33.127974 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:33.128523 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:33.128542 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:33.129510 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:33.129514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:33.130819 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:33.130832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:33.131580 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:33.131588 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:33.131818 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:33.132460 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:33.132488 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:33.135010 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:33.138700 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:33.138707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:33.138720 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:33.139134 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:33.139521 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:33.139524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:33.139526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:33.139716 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:33.140325 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:33.140331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:33.140656 (+ 325us) client_negotiation.cc:769] Sending connection context
0810 06:43:33.140903 (+ 247us) client_negotiation.cc:240] Negotiation successful
0810 06:43:33.141346 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":266,"threads_started":1}
I20260810 06:43:33.144217 12451 heartbeater.cc:347] Connected to a master server at 127.10.242.62:43401
I20260810 06:43:33.144500 12451 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:33.145203 12451 heartbeater.cc:511] Master 127.10.242.62:43401 requested a full tablet report, sending...
I20260810 06:43:33.147467 12125 ts_manager.cc:194] Registered new tserver with Master: 12fe090a834145d3b9161c4b86db16c3 (127.10.242.1:41563)
I20260810 06:43:33.149488 12125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:43229
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:43:33.177094 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:43401
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:33.492640 12460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:33.492898 12460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:33.493000 12460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:33.499950 12460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:33.500044 12460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:33.500159 12460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:43:33.509307 12460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:43401
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:33.510434 12460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:33.511703 12460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:33.523962 12466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:33.523942 12467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:33.523988 12472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:33.524677 12460 server_base.cc:1034] running on GCE node
I20260810 06:43:33.525254 12460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:33.526073 12460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:33.527269 12460 hybrid_clock.cc:648] HybridClock initialized: now 1786344213527223 us; error 41 us; skew 500 ppm
I20260810 06:43:33.527546 12460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:33 dist-test-slave-zj8q krb5kdc[12055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344213, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:33.530416 12460 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:43:33.531719 12460 webserver.cc:460] Webserver started at http://127.10.242.2:44695/ using document root <none> and password file <none>
I20260810 06:43:33.532361 12460 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:33.532472 12460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:33.532757 12460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:33.535468 12460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "24720aaf895a4d5596ad18f3b86d1b8a"
format_stamp: "Formatted at 2026-08-10 06:43:33 on dist-test-slave-zj8q"
server_key: "5ef2977b8540178c8058c3a8c2786513"
server_key_iv: "cbe77b2c9857025720a22fd10e431009"
server_key_version: "encryptionkey@0"
I20260810 06:43:33.536216 12460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "24720aaf895a4d5596ad18f3b86d1b8a"
format_stamp: "Formatted at 2026-08-10 06:43:33 on dist-test-slave-zj8q"
server_key: "5ef2977b8540178c8058c3a8c2786513"
server_key_iv: "cbe77b2c9857025720a22fd10e431009"
server_key_version: "encryptionkey@0"
I20260810 06:43:33.540823 12460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:43:33.544042 12485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:33.545189 12460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:43:33.545336 12460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "24720aaf895a4d5596ad18f3b86d1b8a"
format_stamp: "Formatted at 2026-08-10 06:43:33 on dist-test-slave-zj8q"
server_key: "5ef2977b8540178c8058c3a8c2786513"
server_key_iv: "cbe77b2c9857025720a22fd10e431009"
server_key_version: "encryptionkey@0"
I20260810 06:43:33.545504 12460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:33.560029 12460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:33.563778 12460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:33.564029 12460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:33.564919 12460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:33.566147 12460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:33.566265 12460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:33.566380 12460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:33.566433 12460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:33.594126 12460 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:46385
I20260810 06:43:33.594200 12670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:46385 every 8 connection(s)
I20260810 06:43:33.595372 12460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:43:33.602037 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 12460
I20260810 06:43:33.602238 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:43:33.602736 11208 external_mini_cluster.cc:1428] Setting key 74d8bd51af6a3da6aa72e982e8524f39
Aug 10 06:43:33 dist-test-slave-zj8q krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344213, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:33.614794 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:33.597685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:49805 (local address 127.10.242.62:43401)
0810 06:43:33.597849 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:33.597854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:33.599060 (+ 1206us) server_negotiation.cc:407] Connection header received
0810 06:43:33.600870 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:33.600874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:33.600944 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:33.601039 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:33.602593 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:33.603559 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:33.605453 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:33.606061 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:33.606209 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:33.609285 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:33.609291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:33.609294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:33.609329 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:33.611976 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:33.612750 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:33.612753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:33.612756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:33.612879 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:33.613292 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:33.613294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:33.613297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:33.613527 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:33.613645 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:33.614380 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 06:43:33.614541 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:43:33.615999 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:33.598094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:43401 (local address 127.10.242.2:49805)
0810 06:43:33.598906 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:33.598965 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:33.600627 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:33.601237 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:33.601244 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:33.601770 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:33.602424 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:33.602444 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:33.603696 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:33.603700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:33.605309 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:33.605321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:33.606281 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:33.606286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:33.606465 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:33.607042 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:33.607063 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:33.609092 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:33.612114 (+ 3022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:33.612123 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:33.612136 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:33.612640 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:33.612969 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:33.612974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:33.612978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:33.613204 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:33.613638 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:33.613645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:33.614009 (+ 364us) client_negotiation.cc:769] Sending connection context
0810 06:43:33.614300 (+ 291us) client_negotiation.cc:240] Negotiation successful
0810 06:43:33.614784 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":330,"threads_started":1}
I20260810 06:43:33.617244 12671 heartbeater.cc:347] Connected to a master server at 127.10.242.62:43401
I20260810 06:43:33.617518 12671 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:33.618234 12671 heartbeater.cc:511] Master 127.10.242.62:43401 requested a full tablet report, sending...
I20260810 06:43:33.619655 12125 ts_manager.cc:194] Registered new tserver with Master: 24720aaf895a4d5596ad18f3b86d1b8a (127.10.242.2:46385)
I20260810 06:43:33.620435 12125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:49805
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:43:33.645419 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:43401
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:33.961102 12677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:33.961431 12677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:33.961587 12677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:33.968909 12677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:33.969039 12677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:33.969206 12677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:43:33.978848 12677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:43401
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:33.980100 12677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:33.981429 12677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:33.994352 12684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:33.994431 12687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:33.994411 12685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:33.995110 12677 server_base.cc:1034] running on GCE node
I20260810 06:43:33.995744 12677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:33.996657 12677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:33.997870 12677 hybrid_clock.cc:648] HybridClock initialized: now 1786344213997821 us; error 34 us; skew 500 ppm
I20260810 06:43:33.998203 12677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:33 dist-test-slave-zj8q krb5kdc[12055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344213, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:34.001406 12677 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:43:34.002934 12677 webserver.cc:460] Webserver started at http://127.10.242.3:38821/ using document root <none> and password file <none>
I20260810 06:43:34.003602 12677 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:34.003726 12677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:34.004026 12677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:34.007278 12677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "4e78aba873f348ea87b14121e967efda"
format_stamp: "Formatted at 2026-08-10 06:43:34 on dist-test-slave-zj8q"
server_key: "6090793f2838149fbc26a3c8aa536e6d"
server_key_iv: "cfbe91a38a3f1b3f2148ca051de1235f"
server_key_version: "encryptionkey@0"
I20260810 06:43:34.008129 12677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "4e78aba873f348ea87b14121e967efda"
format_stamp: "Formatted at 2026-08-10 06:43:34 on dist-test-slave-zj8q"
server_key: "6090793f2838149fbc26a3c8aa536e6d"
server_key_iv: "cfbe91a38a3f1b3f2148ca051de1235f"
server_key_version: "encryptionkey@0"
I20260810 06:43:34.013505 12677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:43:34.018085 12700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:34.019881 12677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:43:34.020047 12677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "4e78aba873f348ea87b14121e967efda"
format_stamp: "Formatted at 2026-08-10 06:43:34 on dist-test-slave-zj8q"
server_key: "6090793f2838149fbc26a3c8aa536e6d"
server_key_iv: "cfbe91a38a3f1b3f2148ca051de1235f"
server_key_version: "encryptionkey@0"
I20260810 06:43:34.020270 12677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:34.033602 12677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:34.037608 12677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:34.037849 12677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:34.038822 12677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:34.040155 12677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:34.040227 12677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:34.040318 12677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:34.040360 12677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:34.071136 12677 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:38063
I20260810 06:43:34.071187 12874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:38063 every 8 connection(s)
I20260810 06:43:34.072397 12677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:43:34.076869 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 12677
I20260810 06:43:34.077080 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:43:34.077654 11208 external_mini_cluster.cc:1428] Setting key 4aba531502123eb5960c89e280794447
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344213, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:34.092468 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.074857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:60325 (local address 127.10.242.62:43401)
0810 06:43:34.075012 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.075016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.076155 (+ 1139us) server_negotiation.cc:407] Connection header received
0810 06:43:34.078682 (+ 2527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.078689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.078772 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.078871 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:34.080663 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.081547 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.083250 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.083665 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.083763 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.087055 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:34.087060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:34.087063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:34.087097 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:34.089632 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.090294 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.090297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.090300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.090433 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.090789 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.090792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.090794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.091056 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:34.091172 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.091896 (+ 724us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.092093 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:43:34.093572 12878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.075303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:43401 (local address 127.10.242.3:60325)
0810 06:43:34.075990 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:34.076035 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:34.078347 (+ 2312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:34.079137 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:34.079145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:34.079667 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:34.080454 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.080483 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.081696 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.081700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.083116 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.083128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.083857 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.083861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.084055 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.084650 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:34.084673 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:34.086854 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:34.089752 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:34.089759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:34.089771 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:34.090203 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:34.090518 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:34.090522 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:34.090524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:34.090709 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:34.091172 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:34.091178 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:34.091480 (+ 302us) client_negotiation.cc:769] Sending connection context
0810 06:43:34.091780 (+ 300us) client_negotiation.cc:240] Negotiation successful
0810 06:43:34.092267 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":269,"threads_started":1}
I20260810 06:43:34.094890 12875 heartbeater.cc:347] Connected to a master server at 127.10.242.62:43401
I20260810 06:43:34.095196 12875 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:34.095884 12875 heartbeater.cc:511] Master 127.10.242.62:43401 requested a full tablet report, sending...
I20260810 06:43:34.097201 12125 ts_manager.cc:194] Registered new tserver with Master: 4e78aba873f348ea87b14121e967efda (127.10.242.3:38063)
I20260810 06:43:34.097960 12125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:60325
I20260810 06:43:34.105319 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:43:34.125293 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.111697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.10.242.62:43401)
0810 06:43:34.111912 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.111919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.112231 (+ 312us) server_negotiation.cc:407] Connection header received
0810 06:43:34.112381 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.112384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.112457 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.112529 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:34.113948 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.114719 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.115978 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.116437 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.116538 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.119029 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:34.119036 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:34.119041 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:34.119081 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:34.122434 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.123111 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.123118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.123122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.123287 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.123715 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.123720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.123724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.124140 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:34.124401 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.124726 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.124974 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:43:34.132150 12125 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:43:34.139248 12125 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:43:34.152760 12451 heartbeater.cc:503] Master 127.10.242.62:43401 was elected leader, sending a full tablet report...
I20260810 06:43:34.168764 12892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.151302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:46385 (local address 127.0.0.1:51458)
0810 06:43:34.159099 (+ 7797us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:34.159166 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:34.159354 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:34.160467 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:34.160476 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:34.160511 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:34.160934 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.160948 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.163089 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.163095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.165717 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.165741 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.167821 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.167833 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.168028 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.168080 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:43:34.168216 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 06:43:34.168432 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7539,"thread_start_us":7394,"threads_started":1}
I20260810 06:43:34.169633 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.151491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.10.242.2:46385)
0810 06:43:34.158666 (+ 7175us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.158678 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.159558 (+ 880us) server_negotiation.cc:407] Connection header received
0810 06:43:34.159667 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.159675 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.159945 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.160159 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:34.161151 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.162904 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.165939 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.167337 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.167468 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.167672 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.169261 (+ 1589us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.169393 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6960,"thread_start_us":6781,"threads_started":1}
I20260810 06:43:34.173053 12895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.151481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:41563 (local address 127.0.0.1:51176)
0810 06:43:34.164344 (+ 12863us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:34.164369 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:34.164598 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:34.165326 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:34.165331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:34.165357 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:34.165842 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.165859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.167815 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.167821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.170288 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.170303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.172310 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.172316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.172477 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.172509 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:43:34.172615 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 06:43:34.172736 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12741,"thread_start_us":12600,"threads_started":1}
I20260810 06:43:34.173184 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.152144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.10.242.1:41563)
0810 06:43:34.163685 (+ 11541us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.163695 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.164479 (+ 784us) server_negotiation.cc:407] Connection header received
0810 06:43:34.164794 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.164803 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.165069 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.165242 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:34.166076 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.167647 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.170435 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.171836 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.171964 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.172174 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.172689 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.172857 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11345,"thread_start_us":11552,"threads_started":1}
I20260810 06:43:34.173703 12562 tablet_service.cc:1467] Processing CreateTablet for tablet a90c6b0f32af42d8ac18c6c84a52d51e (DEFAULT_TABLE table=test-table [id=41e123265d3a451c92d1f4081c3e000f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:34.174449 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.165482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38063 (local address 127.0.0.1:50502)
0810 06:43:34.166117 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:34.166141 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:34.166276 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:34.167415 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:34.167420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:34.167444 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:34.167871 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.167888 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.170081 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.170086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.171927 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.171940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.173688 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.173695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.173859 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.173892 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:43:34.173990 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 06:43:34.174110 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":541,"threads_started":1}
I20260810 06:43:34.174552 12897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.159879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.10.242.3:38063)
0810 06:43:34.166849 (+ 6970us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.166857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.166888 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:43:34.166996 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.167002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.167221 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.168089 (+ 868us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:34.168375 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.169940 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.172098 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.173209 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.173342 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.173551 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.174053 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.174246 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6799,"thread_start_us":6633,"threads_started":1}
I20260810 06:43:34.176661 12344 tablet_service.cc:1467] Processing CreateTablet for tablet a90c6b0f32af42d8ac18c6c84a52d51e (DEFAULT_TABLE table=test-table [id=41e123265d3a451c92d1f4081c3e000f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:34.177788 12344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a90c6b0f32af42d8ac18c6c84a52d51e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:34.177559 12771 tablet_service.cc:1467] Processing CreateTablet for tablet a90c6b0f32af42d8ac18c6c84a52d51e (DEFAULT_TABLE table=test-table [id=41e123265d3a451c92d1f4081c3e000f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:34.175489 12562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a90c6b0f32af42d8ac18c6c84a52d51e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:34.178701 12771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a90c6b0f32af42d8ac18c6c84a52d51e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:34.190096 12900 tablet_bootstrap.cc:492] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda: Bootstrap starting.
I20260810 06:43:34.192317 12902 tablet_bootstrap.cc:492] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a: Bootstrap starting.
I20260810 06:43:34.194366 12900 tablet_bootstrap.cc:654] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:34.195147 12901 tablet_bootstrap.cc:492] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3: Bootstrap starting.
I20260810 06:43:34.195748 12900 log.cc:824] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:34.196084 12902 tablet_bootstrap.cc:654] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:34.197134 12902 log.cc:824] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:34.198431 12901 tablet_bootstrap.cc:654] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:34.199118 12900 tablet_bootstrap.cc:492] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda: No bootstrap required, opened a new log
I20260810 06:43:34.199405 12900 ts_tablet_manager.cc:1397] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 06:43:34.199488 12901 log.cc:824] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:34.200189 12902 tablet_bootstrap.cc:492] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a: No bootstrap required, opened a new log
I20260810 06:43:34.200444 12902 ts_tablet_manager.cc:1397] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 06:43:34.204739 12901 tablet_bootstrap.cc:492] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3: No bootstrap required, opened a new log
I20260810 06:43:34.204995 12901 ts_tablet_manager.cc:1397] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 06:43:34.208212 12900 raft_consensus.cc:348] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } }
I20260810 06:43:34.208549 12900 raft_consensus.cc:374] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:34.208648 12900 raft_consensus.cc:729] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e78aba873f348ea87b14121e967efda, State: Initialized, Role: FOLLOWER
I20260810 06:43:34.209206 12900 consensus_queue.cc:260] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } }
I20260810 06:43:34.210250 12875 heartbeater.cc:503] Master 127.10.242.62:43401 was elected leader, sending a full tablet report...
I20260810 06:43:34.210269 12902 raft_consensus.cc:348] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } }
I20260810 06:43:34.210659 12902 raft_consensus.cc:374] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:34.210741 12902 raft_consensus.cc:729] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24720aaf895a4d5596ad18f3b86d1b8a, State: Initialized, Role: FOLLOWER
I20260810 06:43:34.210886 12900 ts_tablet_manager.cc:1428] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260810 06:43:34.211393 12902 consensus_queue.cc:260] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } }
I20260810 06:43:34.212469 12671 heartbeater.cc:503] Master 127.10.242.62:43401 was elected leader, sending a full tablet report...
I20260810 06:43:34.213083 12902 ts_tablet_manager.cc:1428] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260810 06:43:34.216925 12901 raft_consensus.cc:348] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } }
I20260810 06:43:34.217257 12901 raft_consensus.cc:374] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:34.217329 12901 raft_consensus.cc:729] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12fe090a834145d3b9161c4b86db16c3, State: Initialized, Role: FOLLOWER
I20260810 06:43:34.217751 12901 consensus_queue.cc:260] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } }
I20260810 06:43:34.219203 12901 ts_tablet_manager.cc:1428] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 06:43:34.243026 12909 raft_consensus.cc:482] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:43:34.243254 12909 raft_consensus.cc:504] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } }
I20260810 06:43:34.244967 12909 leader_election.cc:288] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24720aaf895a4d5596ad18f3b86d1b8a (127.10.242.2:46385), 4e78aba873f348ea87b14121e967efda (127.10.242.3:38063)
I20260810 06:43:34.252615 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.245275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:60599 (local address 127.10.242.2:46385)
0810 06:43:34.245442 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.245446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.246026 (+ 580us) server_negotiation.cc:407] Connection header received
0810 06:43:34.246272 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.246277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.246354 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.246447 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:34.247196 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.248166 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.250644 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.251774 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.251924 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.251995 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.252285 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.252388 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:43:34.252818 12911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.245275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:46385 (local address 127.10.242.1:60599)
0810 06:43:34.245937 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:34.245956 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:34.246087 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:34.246580 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:34.246585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:34.246611 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:34.247040 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.247051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.248301 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.248305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.250496 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.250511 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.251959 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.251964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.252103 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.252141 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 06:43:34.252247 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 06:43:34.252380 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":1,"spinlock_wait_cycles":25728,"thread_start_us":347,"threads_started":1}
I20260810 06:43:34.252867 12897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.246342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:41495 (local address 127.10.242.3:38063)
0810 06:43:34.246524 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.246529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.246767 (+ 238us) server_negotiation.cc:407] Connection header received
0810 06:43:34.246948 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.246951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.247005 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.247072 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:34.247560 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.248527 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.250841 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.251985 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.252115 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.252168 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.252520 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.252634 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:43:34.252936 12912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.246055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38063 (local address 127.10.242.1:41495)
0810 06:43:34.246676 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:34.246696 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:34.246841 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:34.247111 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:34.247116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:34.247151 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:34.247424 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.247432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.248637 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.248642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.250713 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:34.250724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.252204 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.252210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:34.252371 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.252400 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 06:43:34.252481 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:43:34.252596 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":26,"spinlock_wait_cycles":1280,"thread_start_us":319,"threads_started":1}
I20260810 06:43:34.253842 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a90c6b0f32af42d8ac18c6c84a52d51e" candidate_uuid: "12fe090a834145d3b9161c4b86db16c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" is_pre_election: true
I20260810 06:43:34.253919 12807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a90c6b0f32af42d8ac18c6c84a52d51e" candidate_uuid: "12fe090a834145d3b9161c4b86db16c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e78aba873f348ea87b14121e967efda" is_pre_election: true
I20260810 06:43:34.254335 12596 raft_consensus.cc:2454] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12fe090a834145d3b9161c4b86db16c3 in term 0.
I20260810 06:43:34.254400 12807 raft_consensus.cc:2454] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12fe090a834145d3b9161c4b86db16c3 in term 0.
I20260810 06:43:34.254958 12262 leader_election.cc:302] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12fe090a834145d3b9161c4b86db16c3, 24720aaf895a4d5596ad18f3b86d1b8a; no voters:
I20260810 06:43:34.255287 12909 raft_consensus.cc:2781] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:43:34.255407 12909 raft_consensus.cc:482] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:43:34.255511 12909 raft_consensus.cc:3037] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:34.257643 12909 raft_consensus.cc:504] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } }
I20260810 06:43:34.258230 12909 leader_election.cc:288] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [CANDIDATE]: Term 1 election: Requested vote from peers 24720aaf895a4d5596ad18f3b86d1b8a (127.10.242.2:46385), 4e78aba873f348ea87b14121e967efda (127.10.242.3:38063)
I20260810 06:43:34.258596 12596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a90c6b0f32af42d8ac18c6c84a52d51e" candidate_uuid: "12fe090a834145d3b9161c4b86db16c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24720aaf895a4d5596ad18f3b86d1b8a"
I20260810 06:43:34.258708 12807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a90c6b0f32af42d8ac18c6c84a52d51e" candidate_uuid: "12fe090a834145d3b9161c4b86db16c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e78aba873f348ea87b14121e967efda"
I20260810 06:43:34.258802 12596 raft_consensus.cc:3037] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:34.258895 12807 raft_consensus.cc:3037] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:34.262072 12807 raft_consensus.cc:2454] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12fe090a834145d3b9161c4b86db16c3 in term 1.
I20260810 06:43:34.262077 12596 raft_consensus.cc:2454] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12fe090a834145d3b9161c4b86db16c3 in term 1.
I20260810 06:43:34.262447 12262 leader_election.cc:302] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12fe090a834145d3b9161c4b86db16c3, 24720aaf895a4d5596ad18f3b86d1b8a; no voters:
I20260810 06:43:34.262706 12909 raft_consensus.cc:2781] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:34.263134 12909 raft_consensus.cc:686] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [term 1 LEADER]: Becoming Leader. State: Replica: 12fe090a834145d3b9161c4b86db16c3, State: Running, Role: LEADER
I20260810 06:43:34.263557 12909 consensus_queue.cc:237] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } }
I20260810 06:43:34.267804 12125 catalog_manager.cc:5495] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12fe090a834145d3b9161c4b86db16c3 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "12fe090a834145d3b9161c4b86db16c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24720aaf895a4d5596ad18f3b86d1b8a" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12fe090a834145d3b9161c4b86db16c3" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e78aba873f348ea87b14121e967efda" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38063 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:43:34.301443 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.287233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51184 (local address 127.10.242.1:41563)
0810 06:43:34.287440 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.287448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.287867 (+ 419us) server_negotiation.cc:407] Connection header received
0810 06:43:34.288077 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.288082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.288165 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.288242 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:34.289787 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.290548 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.291751 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.292134 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.292221 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.296131 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:34.296144 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:34.296160 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:34.296194 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:34.298799 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.299486 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.299494 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.299496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.299639 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.299991 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.299995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.299997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.300336 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:34.300614 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.300913 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.301141 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 06:43:34.323999 12876 tablet.cc:2369] T a90c6b0f32af42d8ac18c6c84a52d51e P 4e78aba873f348ea87b14121e967efda: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344214, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344214, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
W20260810 06:43:34.347289 12673 tablet.cc:2369] T a90c6b0f32af42d8ac18c6c84a52d51e P 24720aaf895a4d5596ad18f3b86d1b8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:43:34.349308 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.336148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.10.242.1:41563)
0810 06:43:34.336330 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.336337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.336663 (+ 326us) server_negotiation.cc:407] Connection header received
0810 06:43:34.336846 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.336850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.336917 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.337002 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:34.338402 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.339138 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.340402 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.340839 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.340947 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.344579 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:34.344583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:34.344587 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:34.344621 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:34.346933 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.347625 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.347629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.347631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.347748 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.348179 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.348183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.348185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.348448 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:34.348704 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.348862 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.349059 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 06:43:34.373646 12452 tablet.cc:2369] T a90c6b0f32af42d8ac18c6c84a52d51e P 12fe090a834145d3b9161c4b86db16c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:43:34.394552 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.381324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.10.242.1:41563)
0810 06:43:34.381497 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.381504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.381833 (+ 329us) server_negotiation.cc:407] Connection header received
0810 06:43:34.382018 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.382022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.382101 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.382220 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:34.383614 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.384374 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.385576 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.385982 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.386106 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.389771 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:34.389776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:34.389779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:34.389812 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:34.392056 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.392649 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.392653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.392655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.392786 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.393154 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.393158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.393160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.393445 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:34.393691 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.393956 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.394203 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 06:43:34.396631 12304 tablet_service.cc:2908] Rejecting scan request for tablet a90c6b0f32af42d8ac18c6c84a52d51e: Uninitialized: clean time has not yet been initialized
I20260810 06:43:34.400112 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 12234
I20260810 06:43:34.410878 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.400823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.10.242.1:41563)
0810 06:43:34.400951 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.400955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.401310 (+ 355us) server_negotiation.cc:407] Connection header received
0810 06:43:34.401442 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.401445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.401496 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.401568 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:34.402657 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.403333 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.404389 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.404747 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.404829 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.406720 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:34.406725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:34.406728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:34.406755 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:34.408883 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.409453 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.409457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.409459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.409558 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.409803 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.409806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.409808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.410032 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:34.410241 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.410472 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.410644 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:43:34.411429 12445 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:34.499454 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 12234
I20260810 06:43:34.519428 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 12460
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:43:34.533074 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.520271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.10.242.2:46385)
0810 06:43:34.520423 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.520428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.520445 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:43:34.520504 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.520508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.520569 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.520652 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:34.521773 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.522540 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.523786 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.524151 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.524236 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.528024 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:34.528038 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:34.528053 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:34.528089 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:34.530870 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.531432 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.531438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.531441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.531563 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.531866 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.531869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.531871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.532224 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:34.532456 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.532634 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.532804 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:43:34.533753 12658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:34.618940 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 12460
I20260810 06:43:34.641728 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 12677
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:43:34.656205 12897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:34.642513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.10.242.3:38063)
0810 06:43:34.642717 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:34.642735 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:34.642755 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:43:34.642842 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:34.642845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:34.642914 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:34.643007 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:34.644226 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.645014 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.646468 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:34.646907 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:34.647006 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:34.650948 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:34.650965 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:34.650982 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:34.651023 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:34.653692 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.654343 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.654351 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.654353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.654493 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:34.654860 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:34.654865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:34.654867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:34.655262 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:34.655560 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:34.655703 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 06:43:34.655869 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:43:34.656836 12868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:34.742007 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 12677
I20260810 06:43:34.761234 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 12074
I20260810 06:43:34.762398 12181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:34.872601 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 12074
2026-08-10T06:43:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4123 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12947](info): set up 2 sockets
Aug 10 06:43:34 dist-test-slave-zj8q krb5kdc[12947](info): commencing operation
krb5kdc: starting...
W20260810 06:43:36.089807 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:43:36 dist-test-slave-zj8q krb5kdc[12947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:43:36.280697 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:41351
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:41351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:36.596021 12967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:36.596292 12967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:36.596419 12967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:36.603422 12967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:36.603514 12967 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:43:36.603569 12967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:36.603658 12967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:36.603717 12967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:36.603798 12967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:36.613191 12967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:41351
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:41351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:36.614362 12967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:36.615643 12967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:36.626816 12973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:36.626758 12974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:36.626847 12978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:36.627811 12967 server_base.cc:1034] running on GCE node
I20260810 06:43:36.628485 12967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:36.629777 12967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:36.630990 12967 hybrid_clock.cc:648] HybridClock initialized: now 1786344216630931 us; error 29 us; skew 500 ppm
I20260810 06:43:36.631283 12967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:36 dist-test-slave-zj8q krb5kdc[12947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344216, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:36.634274 12967 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:43:36.635618 12967 webserver.cc:460] Webserver started at http://127.10.242.62:41579/ using document root <none> and password file <none>
I20260810 06:43:36.636252 12967 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:36.636336 12967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:36.636595 12967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:36.639451 12967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "a16c4e50927241aab6a04ecaf7557941"
format_stamp: "Formatted at 2026-08-10 06:43:36 on dist-test-slave-zj8q"
server_key: "18d317ed8ec64655dc6ab4393e56914c"
server_key_iv: "a30c478d70bd0d9aa175c738f4cac5f0"
server_key_version: "encryptionkey@0"
I20260810 06:43:36.640157 12967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "a16c4e50927241aab6a04ecaf7557941"
format_stamp: "Formatted at 2026-08-10 06:43:36 on dist-test-slave-zj8q"
server_key: "18d317ed8ec64655dc6ab4393e56914c"
server_key_iv: "a30c478d70bd0d9aa175c738f4cac5f0"
server_key_version: "encryptionkey@0"
I20260810 06:43:36.645056 12967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:43:36.648463 12990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:36.649685 12967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:43:36.649834 12967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "a16c4e50927241aab6a04ecaf7557941"
format_stamp: "Formatted at 2026-08-10 06:43:36 on dist-test-slave-zj8q"
server_key: "18d317ed8ec64655dc6ab4393e56914c"
server_key_iv: "a30c478d70bd0d9aa175c738f4cac5f0"
server_key_version: "encryptionkey@0"
I20260810 06:43:36.650033 12967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:36.668938 12967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:36.672880 12967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:36.673118 12967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:36.693471 12967 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:41351
I20260810 06:43:36.693491 13075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:41351 every 8 connection(s)
I20260810 06:43:36.694803 12967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:36.695801 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 12967
I20260810 06:43:36.696071 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:43:36.696894 11208 external_mini_cluster.cc:1428] Setting key 32f93dc7a4ec6c7ff6409e13147cbb66
I20260810 06:43:36.699445 13078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:43:36 dist-test-slave-zj8q krb5kdc[12947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:36.711728 13078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941: Bootstrap starting.
I20260810 06:43:36.714726 13078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:36.715682 13078 log.cc:824] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:36.718392 13078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941: No bootstrap required, opened a new log
I20260810 06:43:36.718617 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:36.698913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.10.242.62:41351)
0810 06:43:36.699653 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:36.699665 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:36.699711 (+ 46us) server_negotiation.cc:407] Connection header received
0810 06:43:36.701286 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:36.701305 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:36.701655 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:36.701974 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:36.703665 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:36.705009 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:36.706728 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:36.707369 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:36.707487 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:36.711978 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:36.711985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:36.711998 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:36.712031 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:36.714846 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:36.715470 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:36.715477 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:36.715482 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:36.715648 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:36.716016 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:36.716019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:36.716021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:36.716527 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:36.716873 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:36.717278 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 06:43:36.717651 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":340,"threads_started":1}
I20260810 06:43:36.725832 13078 raft_consensus.cc:348] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16c4e50927241aab6a04ecaf7557941" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41351 } }
I20260810 06:43:36.726195 13078 raft_consensus.cc:374] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:36.726294 13078 raft_consensus.cc:729] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16c4e50927241aab6a04ecaf7557941, State: Initialized, Role: FOLLOWER
I20260810 06:43:36.726794 13078 consensus_queue.cc:260] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16c4e50927241aab6a04ecaf7557941" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41351 } }
I20260810 06:43:36.727005 13078 raft_consensus.cc:388] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:36.727133 13078 raft_consensus.cc:482] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:36.727279 13078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:36.729482 13078 raft_consensus.cc:504] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16c4e50927241aab6a04ecaf7557941" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41351 } }
I20260810 06:43:36.729944 13078 leader_election.cc:302] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a16c4e50927241aab6a04ecaf7557941; no voters:
I20260810 06:43:36.730471 13078 leader_election.cc:288] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:36.730561 13089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:36.731206 13089 raft_consensus.cc:686] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [term 1 LEADER]: Becoming Leader. State: Replica: a16c4e50927241aab6a04ecaf7557941, State: Running, Role: LEADER
I20260810 06:43:36.731654 13089 consensus_queue.cc:237] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16c4e50927241aab6a04ecaf7557941" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41351 } }
I20260810 06:43:36.732267 13078 sys_catalog.cc:564] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:36.734205 13090 sys_catalog.cc:455] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a16c4e50927241aab6a04ecaf7557941" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16c4e50927241aab6a04ecaf7557941" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41351 } } }
I20260810 06:43:36.734612 13090 sys_catalog.cc:458] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:36.735294 13091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a16c4e50927241aab6a04ecaf7557941. Latest consensus state: current_term: 1 leader_uuid: "a16c4e50927241aab6a04ecaf7557941" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16c4e50927241aab6a04ecaf7557941" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41351 } } }
I20260810 06:43:36.735633 13091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:36.736083 13095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:36.741474 13095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:36.750180 13095 catalog_manager.cc:1347] Generated new cluster ID: 9e63117be8534bfcad019cba374244bd
I20260810 06:43:36.750277 13095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:36.762079 13095 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:36.763880 13095 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:36.780689 13095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941: Generated new TSK 0
I20260810 06:43:36.781728 13095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:43:36.835449 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:41351
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:37.166591 13117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:37.166857 13117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:37.166958 13117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:37.174263 13117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:37.174363 13117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:37.174455 13117 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:43:37.174535 13117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:43:37.184391 13117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.62:41351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:37.185640 13117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:37.186930 13117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:37.199299 13126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:37.199352 13131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:37.199381 13127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:37.200346 13117 server_base.cc:1034] running on GCE node
I20260810 06:43:37.200973 13117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:37.201794 13117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:37.203006 13117 hybrid_clock.cc:648] HybridClock initialized: now 1786344217202936 us; error 38 us; skew 500 ppm
I20260810 06:43:37.203291 13117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:37 dist-test-slave-zj8q krb5kdc[12947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344217, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:37.206317 13117 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:43:37.207759 13117 webserver.cc:460] Webserver started at http://127.10.242.1:39789/ using document root <none> and password file <none>
I20260810 06:43:37.208608 13117 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:37.208770 13117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:37.209165 13117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:37.212208 13117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "9e355e6966fc4b3dadbd4bb3e1a07424"
format_stamp: "Formatted at 2026-08-10 06:43:37 on dist-test-slave-zj8q"
server_key: "77ecb45bde345e5b91f687203fca4d1e"
server_key_iv: "d3c4be4c0106d97bfd7ba874d4904cb7"
server_key_version: "encryptionkey@0"
I20260810 06:43:37.212993 13117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "9e355e6966fc4b3dadbd4bb3e1a07424"
format_stamp: "Formatted at 2026-08-10 06:43:37 on dist-test-slave-zj8q"
server_key: "77ecb45bde345e5b91f687203fca4d1e"
server_key_iv: "d3c4be4c0106d97bfd7ba874d4904cb7"
server_key_version: "encryptionkey@0"
I20260810 06:43:37.218138 13117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 06:43:37.221424 13142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:37.222870 13117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 06:43:37.223016 13117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "9e355e6966fc4b3dadbd4bb3e1a07424"
format_stamp: "Formatted at 2026-08-10 06:43:37 on dist-test-slave-zj8q"
server_key: "77ecb45bde345e5b91f687203fca4d1e"
server_key_iv: "d3c4be4c0106d97bfd7ba874d4904cb7"
server_key_version: "encryptionkey@0"
I20260810 06:43:37.223204 13117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:37.258424 13117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:37.262214 13117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:37.262462 13117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:37.265206 13117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:37.265300 13117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:37.265445 13117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:37.265527 13117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:43:37 dist-test-slave-zj8q krb5kdc[12947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344217, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:37.287848 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:37.266342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:35895 (local address 127.10.242.62:41351)
0810 06:43:37.266748 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:37.266753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:37.267659 (+ 906us) server_negotiation.cc:407] Connection header received
0810 06:43:37.269604 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:37.269609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:37.269816 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:37.269945 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:37.272159 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.273090 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.275556 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.276108 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.276224 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:37.280522 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:37.280529 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:37.280534 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:37.280594 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:37.283879 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:37.284880 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:37.284886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:37.284889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:37.285046 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:37.285682 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:37.285687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:37.285692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:37.286026 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:37.286207 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:37.287154 (+ 947us) server_negotiation.cc:299] Negotiation successful
0810 06:43:37.287427 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":218,"threads_started":1}
I20260810 06:43:37.288908 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:37.266695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41351 (local address 127.10.242.1:35895)
0810 06:43:37.267473 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:37.267508 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:37.269337 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:37.270285 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:37.270302 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:37.271091 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:37.271903 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:37.271931 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.273267 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.273276 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:37.275420 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:37.275433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.276373 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.276386 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:37.276609 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:37.277312 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:37.277341 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:37.280203 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:37.284018 (+ 3815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:37.284028 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:37.284048 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:37.284729 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:37.285199 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:37.285202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:37.285205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:37.285509 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:37.286245 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:37.286256 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:37.286750 (+ 494us) client_negotiation.cc:769] Sending connection context
0810 06:43:37.287032 (+ 282us) client_negotiation.cc:240] Negotiation successful
0810 06:43:37.287475 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":344,"threads_started":1}
I20260810 06:43:37.308581 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:37.294662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:55175 (local address 127.10.242.62:41351)
0810 06:43:37.294839 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:37.294844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:37.295770 (+ 926us) server_negotiation.cc:407] Connection header received
0810 06:43:37.295919 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:37.295922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:37.295989 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:37.296095 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:37.297307 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.298435 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.300073 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.300714 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.300840 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:37.302460 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:37.302466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:37.302471 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:37.302509 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:37.305586 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:37.306497 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:37.306503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:37.306508 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:37.306669 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:37.307090 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:37.307095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:37.307098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:37.307435 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:37.307598 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:37.307979 (+ 381us) server_negotiation.cc:299] Negotiation successful
0810 06:43:37.308207 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:43:37.308687 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:37.294451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41351 (local address 127.10.242.1:55175)
0810 06:43:37.295660 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:37.295680 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:37.295822 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:37.296135 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:37.296140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:37.296695 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:37.297111 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:37.297122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.298598 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.298603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:37.299933 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:37.299946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.300913 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.300920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:37.301040 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:37.301567 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:37.301586 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:37.302298 (+ 712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:37.305720 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:37.305726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:37.305730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:37.306323 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:37.306779 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:37.306784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:37.306792 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:37.306943 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:37.307546 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:37.307550 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:37.307705 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 06:43:37.307834 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 06:43:37.308221 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"thread_start_us":862,"threads_started":1}
I20260810 06:43:37.311196 13117 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:40909
I20260810 06:43:37.311230 13332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:40909 every 8 connection(s)
I20260810 06:43:37.312412 13117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:43:37.314261 13333 heartbeater.cc:347] Connected to a master server at 127.10.242.62:41351
I20260810 06:43:37.314471 13333 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:37.315093 13333 heartbeater.cc:511] Master 127.10.242.62:41351 requested a full tablet report, sending...
I20260810 06:43:37.315933 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13117
I20260810 06:43:37.316143 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:43:37.316980 11208 external_mini_cluster.cc:1428] Setting key 5dc69e71f41e7471bbdcad0a15e06734
I20260810 06:43:37.317301 13013 ts_manager.cc:194] Registered new tserver with Master: 9e355e6966fc4b3dadbd4bb3e1a07424 (127.10.242.1:40909)
I20260810 06:43:37.319144 13013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:35895
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:43:37.357151 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:41351
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:37.667927 13338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:37.668170 13338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:37.668305 13338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:37.675151 13338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:37.675251 13338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:37.675371 13338 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:43:37.675457 13338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:43:37.684584 13338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.62:41351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:37.685771 13338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:37.687127 13338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:37.700644 13346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:37.700984 13345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:37.701022 13349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:37.701656 13338 server_base.cc:1034] running on GCE node
I20260810 06:43:37.702320 13338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:37.703212 13338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:37.704419 13338 hybrid_clock.cc:648] HybridClock initialized: now 1786344217704354 us; error 44 us; skew 500 ppm
I20260810 06:43:37.704700 13338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:37 dist-test-slave-zj8q krb5kdc[12947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344217, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:37.711262 13338 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:43:37.712599 13338 webserver.cc:460] Webserver started at http://127.10.242.2:43269/ using document root <none> and password file <none>
I20260810 06:43:37.713246 13338 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:37.713349 13338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:37.713649 13338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:37.716483 13338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "b92af91d8537459081c1a74596721827"
format_stamp: "Formatted at 2026-08-10 06:43:37 on dist-test-slave-zj8q"
server_key: "9e7ae0398f2197405b5ab1e6a7d89975"
server_key_iv: "6481e936f861ef1c4c00bf53807eb4b2"
server_key_version: "encryptionkey@0"
I20260810 06:43:37.717262 13338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "b92af91d8537459081c1a74596721827"
format_stamp: "Formatted at 2026-08-10 06:43:37 on dist-test-slave-zj8q"
server_key: "9e7ae0398f2197405b5ab1e6a7d89975"
server_key_iv: "6481e936f861ef1c4c00bf53807eb4b2"
server_key_version: "encryptionkey@0"
I20260810 06:43:37.722147 13338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260810 06:43:37.725371 13361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:37.726698 13338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:37.726841 13338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "b92af91d8537459081c1a74596721827"
format_stamp: "Formatted at 2026-08-10 06:43:37 on dist-test-slave-zj8q"
server_key: "9e7ae0398f2197405b5ab1e6a7d89975"
server_key_iv: "6481e936f861ef1c4c00bf53807eb4b2"
server_key_version: "encryptionkey@0"
I20260810 06:43:37.727041 13338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:37.765097 13338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:37.769346 13338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:37.769585 13338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:37.772181 13338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:37.772280 13338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:37.772411 13338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:37.772478 13338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:43:37 dist-test-slave-zj8q krb5kdc[12947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344217, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:37.796615 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:37.773820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41351 (local address 127.10.242.2:45425)
0810 06:43:37.774649 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:37.774693 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:37.776468 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:37.777275 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:37.777288 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:37.778035 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:37.778964 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:37.778995 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.780407 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.780413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:37.782835 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:37.782858 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.783738 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.783747 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:37.784028 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:37.784899 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:37.784928 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:37.787925 (+ 2997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:37.791801 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:37.791807 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:37.791819 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:37.792466 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:37.792904 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:37.792908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:37.792912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:37.793234 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:37.793841 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:37.793850 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:37.794208 (+ 358us) client_negotiation.cc:769] Sending connection context
0810 06:43:37.794570 (+ 362us) client_negotiation.cc:240] Negotiation successful
0810 06:43:37.795105 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":371,"threads_started":1}
I20260810 06:43:37.797775 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:37.772854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:45425 (local address 127.10.242.62:41351)
0810 06:43:37.773077 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:37.773085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:37.774806 (+ 1721us) server_negotiation.cc:407] Connection header received
0810 06:43:37.776762 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:37.776768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:37.776878 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:37.776967 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:37.779214 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.780221 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.782999 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.783503 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.783640 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:37.788164 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:37.788171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:37.788184 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:37.788229 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:37.791666 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:37.792592 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:37.792598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:37.792602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:37.792759 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:37.793365 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:37.793371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:37.793374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:37.793706 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:37.793859 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:37.797366 (+ 3507us) server_negotiation.cc:299] Negotiation successful
0810 06:43:37.797535 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:43:37.816213 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:37.802865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:57155 (local address 127.10.242.62:41351)
0810 06:43:37.803031 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:37.803037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:37.803699 (+ 662us) server_negotiation.cc:407] Connection header received
0810 06:43:37.803846 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:37.803849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:37.803904 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:37.803976 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:37.805140 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.805860 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.807833 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.808228 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.808339 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:37.810479 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:37.810486 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:37.810490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:37.810527 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:37.813424 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:37.814107 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:37.814112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:37.814116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:37.814306 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:37.814769 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:37.814774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:37.814777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:37.815080 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:37.815195 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:37.815702 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 06:43:37.815916 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:43:37.816510 13489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:37.802940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41351 (local address 127.10.242.2:57155)
0810 06:43:37.803594 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:37.803614 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:37.803752 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:37.804016 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:37.804020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:37.804621 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:37.805016 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:37.805029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.805973 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.805979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:37.807697 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:37.807713 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:37.808447 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:37.808454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:37.808583 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:37.809145 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:37.809163 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:37.810240 (+ 1077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:37.813542 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:37.813547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:37.813551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:37.814002 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:37.814450 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:37.814455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:37.814459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:37.814650 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:37.815212 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:37.815217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:37.815422 (+ 205us) client_negotiation.cc:769] Sending connection context
0810 06:43:37.815596 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 06:43:37.816008 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":336,"threads_started":1}
I20260810 06:43:37.825734 13338 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:42611
I20260810 06:43:37.825762 13544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:42611 every 8 connection(s)
I20260810 06:43:37.826934 13338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:43:37.828899 13546 heartbeater.cc:347] Connected to a master server at 127.10.242.62:41351
I20260810 06:43:37.829097 13546 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:37.829710 13546 heartbeater.cc:511] Master 127.10.242.62:41351 requested a full tablet report, sending...
I20260810 06:43:37.831022 13013 ts_manager.cc:194] Registered new tserver with Master: b92af91d8537459081c1a74596721827 (127.10.242.2:42611)
I20260810 06:43:37.831861 13013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:45425
I20260810 06:43:37.832836 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13338
I20260810 06:43:37.833020 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:43:37.833446 11208 external_mini_cluster.cc:1428] Setting key b450ca13a50bbd6a71709bcc8df2b35f
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:43:37.871368 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:41351
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:38.199566 13553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:38.199887 13553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:38.200043 13553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:38.207240 13553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:38.207355 13553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:38.207471 13553 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:43:38.207578 13553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:43:38.217077 13553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.62:41351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:38.218377 13553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:38.219726 13553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:38.232255 13562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:38.232210 13564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:38.232286 13566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:38.233145 13553 server_base.cc:1034] running on GCE node
I20260810 06:43:38.233675 13553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:38.234531 13553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:38.235750 13553 hybrid_clock.cc:648] HybridClock initialized: now 1786344218235679 us; error 45 us; skew 500 ppm
I20260810 06:43:38.236012 13553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:38 dist-test-slave-zj8q krb5kdc[12947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344218, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:38.239193 13553 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:43:38.240581 13553 webserver.cc:460] Webserver started at http://127.10.242.3:37209/ using document root <none> and password file <none>
I20260810 06:43:38.241248 13553 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:38.241353 13553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:38.241657 13553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:38.244731 13553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "c3abb014a3c14d20857c275f93d5ac40"
format_stamp: "Formatted at 2026-08-10 06:43:38 on dist-test-slave-zj8q"
server_key: "e67f348e31031d5934eab4733d1a715e"
server_key_iv: "626d48e378ab1a9af82ec5bf3b71d32e"
server_key_version: "encryptionkey@0"
I20260810 06:43:38.245488 13553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "c3abb014a3c14d20857c275f93d5ac40"
format_stamp: "Formatted at 2026-08-10 06:43:38 on dist-test-slave-zj8q"
server_key: "e67f348e31031d5934eab4733d1a715e"
server_key_iv: "626d48e378ab1a9af82ec5bf3b71d32e"
server_key_version: "encryptionkey@0"
I20260810 06:43:38.250347 13553 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260810 06:43:38.253595 13577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:38.254876 13553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:38.255020 13553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "c3abb014a3c14d20857c275f93d5ac40"
format_stamp: "Formatted at 2026-08-10 06:43:38 on dist-test-slave-zj8q"
server_key: "e67f348e31031d5934eab4733d1a715e"
server_key_iv: "626d48e378ab1a9af82ec5bf3b71d32e"
server_key_version: "encryptionkey@0"
I20260810 06:43:38.255226 13553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:38.266718 13553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:38.270320 13553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:38.270562 13553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:38.273097 13553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:38.273205 13553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:38.273368 13553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:38.273504 13553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 10 06:43:38 dist-test-slave-zj8q krb5kdc[12947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344218, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:38.295130 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.273450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:34589 (local address 127.10.242.62:41351)
0810 06:43:38.273639 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:38.273645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:38.275069 (+ 1424us) server_negotiation.cc:407] Connection header received
0810 06:43:38.277138 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:38.277144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:38.277238 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:38.277360 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:38.279479 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.280734 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.283079 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.283743 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.283875 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.288646 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:38.288651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:38.288655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:38.288689 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:38.291666 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:38.292505 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:38.292509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:38.292511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:38.292757 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:38.293278 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:38.293282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:38.293284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:38.293573 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:38.293748 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:38.294536 (+ 788us) server_negotiation.cc:299] Negotiation successful
0810 06:43:38.294783 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:43:38.296095 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.274109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41351 (local address 127.10.242.3:34589)
0810 06:43:38.274915 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:38.274952 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:38.276783 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:38.277625 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:38.277635 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:38.278495 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:38.279272 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.279300 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.280917 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.280922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.282937 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.282950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.284009 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.284017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.284223 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.285010 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:38.285046 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:38.288375 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:38.291798 (+ 3423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:38.291810 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:38.291826 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:38.292378 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:38.292907 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:38.292912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:38.292916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:38.293156 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:38.293734 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:38.293741 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:38.294079 (+ 338us) client_negotiation.cc:769] Sending connection context
0810 06:43:38.294387 (+ 308us) client_negotiation.cc:240] Negotiation successful
0810 06:43:38.294906 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":340,"threads_started":1}
I20260810 06:43:38.319617 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.304963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:58669 (local address 127.10.242.62:41351)
0810 06:43:38.305151 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:38.305158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:38.305709 (+ 551us) server_negotiation.cc:407] Connection header received
0810 06:43:38.305871 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:38.305874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:38.305943 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:38.306020 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:38.307228 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.307915 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.309863 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.310504 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.310619 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.312974 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:38.312982 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:38.312987 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:38.313031 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:38.316543 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:38.317440 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:38.317447 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:38.317451 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:38.317619 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:38.318085 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:38.318090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:38.318094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:38.318419 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:38.318570 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:38.319046 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 06:43:38.319263 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:43:38.319741 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.304959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41351 (local address 127.10.242.3:58669)
0810 06:43:38.305601 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:38.305622 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:38.305774 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:38.306057 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:38.306062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:38.306708 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:38.307093 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.307106 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.308024 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.308029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.309704 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.309721 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.310865 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.310872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.311049 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.311717 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:38.311739 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:38.312745 (+ 1006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:38.316718 (+ 3973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:38.316726 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:38.316730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:38.317314 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:38.317746 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:38.317751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:38.317755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:38.317959 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:38.318596 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:38.318601 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:38.318782 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 06:43:38.318917 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 06:43:38.319272 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":326,"threads_started":1}
I20260810 06:43:38.321808 13333 heartbeater.cc:503] Master 127.10.242.62:41351 was elected leader, sending a full tablet report...
I20260810 06:43:38.332298 13553 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:39285
I20260810 06:43:38.332315 13773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:39285 every 8 connection(s)
I20260810 06:43:38.333470 13553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:43:38.335322 13774 heartbeater.cc:347] Connected to a master server at 127.10.242.62:41351
I20260810 06:43:38.335522 13774 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:38.336117 13774 heartbeater.cc:511] Master 127.10.242.62:41351 requested a full tablet report, sending...
I20260810 06:43:38.337411 13013 ts_manager.cc:194] Registered new tserver with Master: c3abb014a3c14d20857c275f93d5ac40 (127.10.242.3:39285)
I20260810 06:43:38.338287 13013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:34589
I20260810 06:43:38.341413 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13553
I20260810 06:43:38.341562 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:43:38.341944 11208 external_mini_cluster.cc:1428] Setting key cc551ea41b2937731ec09e5917305b74
I20260810 06:43:38.344544 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:43:38 dist-test-slave-zj8q krb5kdc[12947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344218, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:43:38 dist-test-slave-zj8q krb5kdc[12947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344218, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:38.390143 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.377828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.10.242.62:41351)
0810 06:43:38.377984 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:38.377989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:38.378312 (+ 323us) server_negotiation.cc:407] Connection header received
0810 06:43:38.378487 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:38.378491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:38.378557 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:38.378634 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:38.379878 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.380614 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.381784 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.382241 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.382340 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.385853 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:38.385858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:38.385861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:38.385893 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:38.388122 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:38.388648 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:38.388651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:38.388654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:38.388784 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:38.389080 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:38.389083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:38.389084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:38.389324 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:38.389502 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:38.389701 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:43:38.389859 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:43:38.393894 13013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:43:38.400777 13013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:43:38.421721 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.412004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.10.242.2:42611)
0810 06:43:38.413464 (+ 1460us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:38.413471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:38.413493 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:43:38.413566 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:38.413571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:38.413801 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:38.414667 (+ 866us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:38.414860 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.416295 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.419136 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.420467 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.420611 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.420828 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:38.421268 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 06:43:38.421431 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1323,"thread_start_us":1195,"threads_started":1}
I20260810 06:43:38.422031 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.411795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:42611 (local address 127.0.0.1:35988)
0810 06:43:38.412726 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:38.412779 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:38.412957 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:38.414039 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:38.414047 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:38.414078 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:38.414461 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.414476 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.416437 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.416442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.418951 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.418965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.420670 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.420676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.420957 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.421088 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 06:43:38.421204 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 06:43:38.421388 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":86,"thread_start_us":438,"threads_started":1}
I20260810 06:43:38.422081 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.412996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:40909 (local address 127.0.0.1:42968)
0810 06:43:38.413441 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:38.413457 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:38.413562 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:38.414865 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:38.414870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:38.414894 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:38.415313 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.415326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.417022 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.417028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.419220 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.419233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.420950 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.420956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.421152 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.421185 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:43:38.421269 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:43:38.421392 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":2,"thread_start_us":664,"threads_started":1}
I20260810 06:43:38.422336 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.413483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.10.242.1:40909)
0810 06:43:38.414012 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:38.414020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:38.414045 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:43:38.414172 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:38.414181 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:38.414417 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:38.414617 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:38.415502 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.416888 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.419384 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.420742 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.420875 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.421026 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:38.421875 (+ 849us) server_negotiation.cc:299] Negotiation successful
0810 06:43:38.422039 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":1024,"threads_started":1}
I20260810 06:43:38.424381 13801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.413928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:39285 (local address 127.0.0.1:54168)
0810 06:43:38.416902 (+ 2974us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:38.416918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:38.417027 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:38.417496 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:38.417500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:38.417523 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:38.417960 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.417970 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.419732 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.419737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.421445 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.421459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.423608 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.423615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.423785 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.423817 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:43:38.423902 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:43:38.424019 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2853,"mutex_wait_us":148,"thread_start_us":2700,"threads_started":1}
I20260810 06:43:38.424417 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.414330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.10.242.3:39285)
0810 06:43:38.416006 (+ 1676us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:38.416037 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:38.417076 (+ 1039us) server_negotiation.cc:407] Connection header received
0810 06:43:38.417182 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:38.417188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:38.417336 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:38.417484 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:38.418176 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.419602 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.421628 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.423136 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.423260 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.423470 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:38.423972 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 06:43:38.424122 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":360,"threads_started":1}
I20260810 06:43:38.426018 13218 tablet_service.cc:1467] Processing CreateTablet for tablet 420cea73075c4a9ba007cfc520df5d40 (DEFAULT_TABLE table=test-table [id=9b7f6bda150a45f68d8ecf969aa345c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:38.426018 13442 tablet_service.cc:1467] Processing CreateTablet for tablet 420cea73075c4a9ba007cfc520df5d40 (DEFAULT_TABLE table=test-table [id=9b7f6bda150a45f68d8ecf969aa345c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:38.427474 13218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420cea73075c4a9ba007cfc520df5d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:38.427488 13442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420cea73075c4a9ba007cfc520df5d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:38.427609 13654 tablet_service.cc:1467] Processing CreateTablet for tablet 420cea73075c4a9ba007cfc520df5d40 (DEFAULT_TABLE table=test-table [id=9b7f6bda150a45f68d8ecf969aa345c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:38.428933 13654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 420cea73075c4a9ba007cfc520df5d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:38.440127 13805 tablet_bootstrap.cc:492] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: Bootstrap starting.
I20260810 06:43:38.442835 13810 tablet_bootstrap.cc:492] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: Bootstrap starting.
I20260810 06:43:38.443593 13806 tablet_bootstrap.cc:492] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: Bootstrap starting.
I20260810 06:43:38.444681 13805 tablet_bootstrap.cc:654] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:38.445971 13805 log.cc:824] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:38.446979 13806 tablet_bootstrap.cc:654] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:38.448154 13806 log.cc:824] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:38.448943 13810 tablet_bootstrap.cc:654] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:38.449245 13805 tablet_bootstrap.cc:492] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: No bootstrap required, opened a new log
I20260810 06:43:38.449558 13805 ts_tablet_manager.cc:1397] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260810 06:43:38.450109 13810 log.cc:824] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:38.451488 13806 tablet_bootstrap.cc:492] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: No bootstrap required, opened a new log
I20260810 06:43:38.451742 13806 ts_tablet_manager.cc:1397] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 06:43:38.452386 13810 tablet_bootstrap.cc:492] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: No bootstrap required, opened a new log
I20260810 06:43:38.452605 13810 ts_tablet_manager.cc:1397] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260810 06:43:38.459369 13810 raft_consensus.cc:348] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.459720 13810 raft_consensus.cc:374] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:38.459818 13810 raft_consensus.cc:729] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92af91d8537459081c1a74596721827, State: Initialized, Role: FOLLOWER
I20260810 06:43:38.459740 13805 raft_consensus.cc:348] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.460218 13805 raft_consensus.cc:374] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:38.460357 13805 raft_consensus.cc:729] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3abb014a3c14d20857c275f93d5ac40, State: Initialized, Role: FOLLOWER
I20260810 06:43:38.460561 13810 consensus_queue.cc:260] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.461046 13805 consensus_queue.cc:260] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.461613 13546 heartbeater.cc:503] Master 127.10.242.62:41351 was elected leader, sending a full tablet report...
I20260810 06:43:38.462074 13810 ts_tablet_manager.cc:1428] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: Time spent starting tablet: real 0.009s user 0.005s sys 0.005s
I20260810 06:43:38.462069 13806 raft_consensus.cc:348] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.462503 13806 raft_consensus.cc:374] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:38.462603 13806 raft_consensus.cc:729] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e355e6966fc4b3dadbd4bb3e1a07424, State: Initialized, Role: FOLLOWER
I20260810 06:43:38.463181 13806 consensus_queue.cc:260] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.463635 13774 heartbeater.cc:503] Master 127.10.242.62:41351 was elected leader, sending a full tablet report...
I20260810 06:43:38.464217 13805 ts_tablet_manager.cc:1428] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260810 06:43:38.471113 13806 ts_tablet_manager.cc:1428] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
I20260810 06:43:38.475368 13820 raft_consensus.cc:482] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:43:38.475574 13820 raft_consensus.cc:504] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.476923 13820 leader_election.cc:288] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b92af91d8537459081c1a74596721827 (127.10.242.2:42611), c3abb014a3c14d20857c275f93d5ac40 (127.10.242.3:39285)
I20260810 06:43:38.482980 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.477313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:50191 (local address 127.10.242.2:42611)
0810 06:43:38.477475 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:38.477479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:38.478006 (+ 527us) server_negotiation.cc:407] Connection header received
0810 06:43:38.478259 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:38.478264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:38.478334 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:38.478415 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:38.479011 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.479718 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.481414 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.482136 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.482323 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.482396 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:38.482630 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 06:43:38.482754 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:43:38.483048 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.477266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:42611 (local address 127.10.242.1:50191)
0810 06:43:38.477920 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:38.477939 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:38.478071 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:38.478451 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:38.478456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:38.478482 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:38.478878 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.478889 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.479840 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.479844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.481263 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.481275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.482356 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.482362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.482492 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.482515 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:43:38.482595 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:43:38.482722 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":360,"threads_started":1}
I20260810 06:43:38.483913 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.478250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:60619 (local address 127.10.242.3:39285)
0810 06:43:38.478443 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:38.478450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:38.479006 (+ 556us) server_negotiation.cc:407] Connection header received
0810 06:43:38.479167 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:38.479172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:38.479227 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:38.479296 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:38.479786 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.480464 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.482037 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.482840 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.482984 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.483049 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:38.483563 (+ 514us) server_negotiation.cc:299] Negotiation successful
0810 06:43:38.483681 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:43:38.483960 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.478077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:39285 (local address 127.10.242.1:60619)
0810 06:43:38.478910 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:38.478927 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:38.479038 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:38.479321 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:38.479324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:38.479348 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:38.479652 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.479661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.480597 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.480602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.481897 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:38.481909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.483290 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.483300 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:38.483417 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.483440 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:43:38.483521 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:43:38.483645 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":514,"threads_started":1}
I20260810 06:43:38.483870 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420cea73075c4a9ba007cfc520df5d40" candidate_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92af91d8537459081c1a74596721827" is_pre_election: true
I20260810 06:43:38.484308 13475 raft_consensus.cc:2454] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e355e6966fc4b3dadbd4bb3e1a07424 in term 0.
I20260810 06:43:38.484757 13687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420cea73075c4a9ba007cfc520df5d40" candidate_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3abb014a3c14d20857c275f93d5ac40" is_pre_election: true
I20260810 06:43:38.484905 13147 leader_election.cc:302] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e355e6966fc4b3dadbd4bb3e1a07424, b92af91d8537459081c1a74596721827; no voters:
I20260810 06:43:38.485261 13820 raft_consensus.cc:2781] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:43:38.485260 13687 raft_consensus.cc:2454] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e355e6966fc4b3dadbd4bb3e1a07424 in term 0.
I20260810 06:43:38.485380 13820 raft_consensus.cc:482] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:43:38.485481 13820 raft_consensus.cc:3037] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:38.487663 13820 raft_consensus.cc:504] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.488190 13820 leader_election.cc:288] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [CANDIDATE]: Term 1 election: Requested vote from peers b92af91d8537459081c1a74596721827 (127.10.242.2:42611), c3abb014a3c14d20857c275f93d5ac40 (127.10.242.3:39285)
I20260810 06:43:38.488581 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420cea73075c4a9ba007cfc520df5d40" candidate_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92af91d8537459081c1a74596721827"
I20260810 06:43:38.488655 13687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "420cea73075c4a9ba007cfc520df5d40" candidate_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3abb014a3c14d20857c275f93d5ac40"
I20260810 06:43:38.488772 13475 raft_consensus.cc:3037] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:38.488823 13687 raft_consensus.cc:3037] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:38.491027 13475 raft_consensus.cc:2454] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e355e6966fc4b3dadbd4bb3e1a07424 in term 1.
I20260810 06:43:38.491220 13687 raft_consensus.cc:2454] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e355e6966fc4b3dadbd4bb3e1a07424 in term 1.
I20260810 06:43:38.491376 13147 leader_election.cc:302] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e355e6966fc4b3dadbd4bb3e1a07424, b92af91d8537459081c1a74596721827; no voters:
I20260810 06:43:38.491665 13820 raft_consensus.cc:2781] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:38.492053 13820 raft_consensus.cc:686] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 LEADER]: Becoming Leader. State: Replica: 9e355e6966fc4b3dadbd4bb3e1a07424, State: Running, Role: LEADER
I20260810 06:43:38.492503 13820 consensus_queue.cc:237] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.496419 13013 catalog_manager.cc:5495] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e355e6966fc4b3dadbd4bb3e1a07424 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } health_report { overall_health: UNKNOWN } } }
I20260810 06:43:38.523933 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:38.519212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.10.242.1:40909)
0810 06:43:38.519372 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:38.519377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:38.519540 (+ 163us) server_negotiation.cc:407] Connection header received
0810 06:43:38.519797 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:38.519801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:38.519893 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:38.520003 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:38.520273 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.520955 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.522212 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:38.522592 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:38.522683 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:38.523079 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:38.523517 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:38.523613 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:38.523684 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 06:43:38.523744 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:43:38.531191 13687 raft_consensus.cc:1261] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Refusing update from remote peer 9e355e6966fc4b3dadbd4bb3e1a07424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:38.531244 13475 raft_consensus.cc:1261] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Refusing update from remote peer 9e355e6966fc4b3dadbd4bb3e1a07424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:38.532037 13823 consensus_queue.cc:1035] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:38.532394 13820 consensus_queue.cc:1035] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:38.545298 13830 mvcc.cc:205] Tried to move back new op lower bound from 7316865919094947840 to 7316865918948548608. Current Snapshot: MvccSnapshot[applied={T|T < 7316865919094947840}]
I20260810 06:43:38.565105 13013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:38.565366 13013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:38.568703 13013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941: Sending DeleteTablet for 3 replicas of tablet 420cea73075c4a9ba007cfc520df5d40
I20260810 06:43:38.569825 13442 tablet_service.cc:1514] Processing DeleteTablet for tablet 420cea73075c4a9ba007cfc520df5d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:38 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:35988
I20260810 06:43:38.569947 13218 tablet_service.cc:1514] Processing DeleteTablet for tablet 420cea73075c4a9ba007cfc520df5d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:38 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:42968
I20260810 06:43:38.570398 13654 tablet_service.cc:1514] Processing DeleteTablet for tablet 420cea73075c4a9ba007cfc520df5d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:38 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:54168
I20260810 06:43:38.571575 13013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:43:38.572283 13013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:43:38.580004 13217 tablet_service.cc:1467] Processing CreateTablet for tablet c14abd53c16a44968bc747f268a693f0 (DEFAULT_TABLE table=test-table [id=2c107281d3504001a5954d9c542711f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:38.580655 13217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14abd53c16a44968bc747f268a693f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:38.587149 13653 tablet_service.cc:1467] Processing CreateTablet for tablet c14abd53c16a44968bc747f268a693f0 (DEFAULT_TABLE table=test-table [id=2c107281d3504001a5954d9c542711f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:38.587777 13653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14abd53c16a44968bc747f268a693f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:38.590806 13854 tablet_replica.cc:330] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: stopping tablet replica
I20260810 06:43:38.591147 13440 tablet_service.cc:1467] Processing CreateTablet for tablet c14abd53c16a44968bc747f268a693f0 (DEFAULT_TABLE table=test-table [id=2c107281d3504001a5954d9c542711f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:38.591257 13854 raft_consensus.cc:2229] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:38.591768 13440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c14abd53c16a44968bc747f268a693f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:38.591805 13854 raft_consensus.cc:2258] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:38.594172 13854 ts_tablet_manager.cc:1905] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:38.599609 13810 tablet_bootstrap.cc:492] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: Bootstrap starting.
I20260810 06:43:38.600284 13806 tablet_bootstrap.cc:492] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: Bootstrap starting.
I20260810 06:43:38.601225 13810 tablet_bootstrap.cc:654] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:38.601879 13806 tablet_bootstrap.cc:654] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:38.602710 13805 tablet_bootstrap.cc:492] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: Bootstrap starting.
I20260810 06:43:38.604364 13805 tablet_bootstrap.cc:654] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:38.604717 13854 ts_tablet_manager.cc:1918] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:43:38.604895 13854 log.cc:1196] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/420cea73075c4a9ba007cfc520df5d40
I20260810 06:43:38.605906 13854 ts_tablet_manager.cc:1939] T 420cea73075c4a9ba007cfc520df5d40 P c3abb014a3c14d20857c275f93d5ac40: Deleting consensus metadata
I20260810 06:43:38.607797 12995 catalog_manager.cc:4841] TS c3abb014a3c14d20857c275f93d5ac40 (127.10.242.3:39285): tablet 420cea73075c4a9ba007cfc520df5d40 (table test-table [id=9b7f6bda150a45f68d8ecf969aa345c4]) successfully deleted
I20260810 06:43:38.610491 13810 tablet_bootstrap.cc:492] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: No bootstrap required, opened a new log
I20260810 06:43:38.610644 13810 ts_tablet_manager.cc:1397] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260810 06:43:38.611541 13810 raft_consensus.cc:348] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.611783 13810 raft_consensus.cc:374] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:38.611873 13810 raft_consensus.cc:729] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92af91d8537459081c1a74596721827, State: Initialized, Role: FOLLOWER
I20260810 06:43:38.612226 13810 consensus_queue.cc:260] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.612836 13810 ts_tablet_manager.cc:1428] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:43:38.615022 13806 tablet_bootstrap.cc:492] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: No bootstrap required, opened a new log
I20260810 06:43:38.615190 13806 ts_tablet_manager.cc:1397] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.007s
I20260810 06:43:38.616075 13806 raft_consensus.cc:348] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.616308 13806 raft_consensus.cc:374] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:38.616403 13806 raft_consensus.cc:729] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e355e6966fc4b3dadbd4bb3e1a07424, State: Initialized, Role: FOLLOWER
I20260810 06:43:38.616674 13806 consensus_queue.cc:260] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.617364 13806 ts_tablet_manager.cc:1428] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 06:43:38.618016 13850 tablet_replica.cc:330] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: stopping tablet replica
I20260810 06:43:38.618511 13850 raft_consensus.cc:2229] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:43:38.619153 13850 raft_consensus.cc:2258] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:38.619980 13853 tablet_replica.cc:330] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: stopping tablet replica
I20260810 06:43:38.620376 13853 raft_consensus.cc:2229] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:38.620858 13853 raft_consensus.cc:2258] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:38.621902 13850 ts_tablet_manager.cc:1905] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:38.622802 13805 tablet_bootstrap.cc:492] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: No bootstrap required, opened a new log
I20260810 06:43:38.622859 13853 ts_tablet_manager.cc:1905] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:38.622957 13805 ts_tablet_manager.cc:1397] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.006s
I20260810 06:43:38.623777 13805 raft_consensus.cc:348] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.623997 13805 raft_consensus.cc:374] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:38.624099 13805 raft_consensus.cc:729] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3abb014a3c14d20857c275f93d5ac40, State: Initialized, Role: FOLLOWER
I20260810 06:43:38.624358 13805 consensus_queue.cc:260] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:38.624943 13805 ts_tablet_manager.cc:1428] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:43:38.629910 13853 ts_tablet_manager.cc:1918] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:43:38.630170 13853 log.cc:1196] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/420cea73075c4a9ba007cfc520df5d40
I20260810 06:43:38.630726 13853 ts_tablet_manager.cc:1939] T 420cea73075c4a9ba007cfc520df5d40 P b92af91d8537459081c1a74596721827: Deleting consensus metadata
I20260810 06:43:38.630998 13850 ts_tablet_manager.cc:1918] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:43:38.631177 13850 log.cc:1196] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/420cea73075c4a9ba007cfc520df5d40
I20260810 06:43:38.631736 13850 ts_tablet_manager.cc:1939] T 420cea73075c4a9ba007cfc520df5d40 P 9e355e6966fc4b3dadbd4bb3e1a07424: Deleting consensus metadata
I20260810 06:43:38.632637 12995 catalog_manager.cc:4841] TS b92af91d8537459081c1a74596721827 (127.10.242.2:42611): tablet 420cea73075c4a9ba007cfc520df5d40 (table test-table [id=9b7f6bda150a45f68d8ecf969aa345c4]) successfully deleted
I20260810 06:43:38.633340 12995 catalog_manager.cc:4841] TS 9e355e6966fc4b3dadbd4bb3e1a07424 (127.10.242.1:40909): tablet 420cea73075c4a9ba007cfc520df5d40 (table test-table [id=9b7f6bda150a45f68d8ecf969aa345c4]) successfully deleted
W20260810 06:43:38.815577 13335 tablet.cc:2369] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:43:38.829923 13548 tablet.cc:2369] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:43:38.835990 13776 tablet.cc:2369] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:43:39.031986 13823 raft_consensus.cc:482] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:43:39.032191 13823 raft_consensus.cc:504] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:39.032793 13823 leader_election.cc:288] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b92af91d8537459081c1a74596721827 (127.10.242.2:42611), c3abb014a3c14d20857c275f93d5ac40 (127.10.242.3:39285)
I20260810 06:43:39.033524 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14abd53c16a44968bc747f268a693f0" candidate_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92af91d8537459081c1a74596721827" is_pre_election: true
I20260810 06:43:39.033500 13687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14abd53c16a44968bc747f268a693f0" candidate_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3abb014a3c14d20857c275f93d5ac40" is_pre_election: true
I20260810 06:43:39.033802 13687 raft_consensus.cc:2454] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e355e6966fc4b3dadbd4bb3e1a07424 in term 0.
I20260810 06:43:39.033802 13475 raft_consensus.cc:2454] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e355e6966fc4b3dadbd4bb3e1a07424 in term 0.
I20260810 06:43:39.034250 13147 leader_election.cc:302] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e355e6966fc4b3dadbd4bb3e1a07424, c3abb014a3c14d20857c275f93d5ac40; no voters:
I20260810 06:43:39.034510 13823 raft_consensus.cc:2781] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:43:39.034612 13823 raft_consensus.cc:482] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:43:39.034745 13823 raft_consensus.cc:3037] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:39.036741 13823 raft_consensus.cc:504] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:39.037245 13823 leader_election.cc:288] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [CANDIDATE]: Term 1 election: Requested vote from peers b92af91d8537459081c1a74596721827 (127.10.242.2:42611), c3abb014a3c14d20857c275f93d5ac40 (127.10.242.3:39285)
I20260810 06:43:39.037755 13687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14abd53c16a44968bc747f268a693f0" candidate_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3abb014a3c14d20857c275f93d5ac40"
I20260810 06:43:39.037776 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c14abd53c16a44968bc747f268a693f0" candidate_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92af91d8537459081c1a74596721827"
I20260810 06:43:39.037969 13475 raft_consensus.cc:3037] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:39.037968 13687 raft_consensus.cc:3037] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:39.040805 13475 raft_consensus.cc:2454] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e355e6966fc4b3dadbd4bb3e1a07424 in term 1.
I20260810 06:43:39.040805 13687 raft_consensus.cc:2454] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e355e6966fc4b3dadbd4bb3e1a07424 in term 1.
I20260810 06:43:39.041163 13147 leader_election.cc:302] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e355e6966fc4b3dadbd4bb3e1a07424, c3abb014a3c14d20857c275f93d5ac40; no voters:
I20260810 06:43:39.041399 13823 raft_consensus.cc:2781] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:39.041554 13823 raft_consensus.cc:686] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 LEADER]: Becoming Leader. State: Replica: 9e355e6966fc4b3dadbd4bb3e1a07424, State: Running, Role: LEADER
I20260810 06:43:39.041781 13823 consensus_queue.cc:237] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } }
I20260810 06:43:39.044245 13013 catalog_manager.cc:5495] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e355e6966fc4b3dadbd4bb3e1a07424 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:43:39 dist-test-slave-zj8q krb5kdc[12947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344216, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:39.185111 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.172314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41351 (local address 127.0.0.1:37102)
0810 06:43:39.173010 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:39.173026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:39.173150 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:39.173432 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:39.173436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:39.174112 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:39.174348 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.174358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.175496 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.175500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.176499 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.176512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.177193 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.177197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.177315 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.177913 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:39.177934 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:39.180101 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:39.182938 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:39.182950 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:39.182956 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:39.183386 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:39.183677 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:39.183680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:39.183682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:39.183847 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:39.184289 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:39.184295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:39.184449 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 06:43:39.184584 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 06:43:39.184799 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":394,"threads_started":1}
I20260810 06:43:39.185214 13870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.172323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.10.242.62:41351)
0810 06:43:39.172941 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.172946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.173097 (+ 151us) server_negotiation.cc:407] Connection header received
0810 06:43:39.173240 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.173243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.173302 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.173391 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:39.174534 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.175381 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.176633 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.177055 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.177143 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.180266 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:39.180271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:39.180274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:39.180302 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:39.182823 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.183480 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.183484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.183486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.183600 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.183929 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.183932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.183933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.184188 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:39.184290 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.184664 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.184903 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":497,"thread_start_us":357,"threads_started":1}
I20260810 06:43:39.190004 13013 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:37102:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:43:39.191057 13013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:43:39.197355 13218 tablet_service.cc:1467] Processing CreateTablet for tablet 3f5b74e0c579457f9a727d0f299f02d8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5471fcc4b7974d7992b5f5644b83542d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:43:39.197569 13653 tablet_service.cc:1467] Processing CreateTablet for tablet 3f5b74e0c579457f9a727d0f299f02d8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5471fcc4b7974d7992b5f5644b83542d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:43:39.197947 13218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f5b74e0c579457f9a727d0f299f02d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:39.198084 13653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f5b74e0c579457f9a727d0f299f02d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:39.198589 13442 tablet_service.cc:1467] Processing CreateTablet for tablet 3f5b74e0c579457f9a727d0f299f02d8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5471fcc4b7974d7992b5f5644b83542d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:43:39.199096 13442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f5b74e0c579457f9a727d0f299f02d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:39.207605 13878 tablet_bootstrap.cc:492] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: Bootstrap starting.
I20260810 06:43:39.208233 13877 tablet_bootstrap.cc:492] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: Bootstrap starting.
I20260810 06:43:39.209357 13878 tablet_bootstrap.cc:654] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:39.209918 13877 tablet_bootstrap.cc:654] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:39.211647 13879 tablet_bootstrap.cc:492] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: Bootstrap starting.
I20260810 06:43:39.211979 13878 tablet_bootstrap.cc:492] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: No bootstrap required, opened a new log
I20260810 06:43:39.212136 13878 ts_tablet_manager.cc:1397] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260810 06:43:39.212867 13878 raft_consensus.cc:348] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } }
I20260810 06:43:39.213132 13878 raft_consensus.cc:374] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:39.213201 13879 tablet_bootstrap.cc:654] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:39.213238 13878 raft_consensus.cc:729] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92af91d8537459081c1a74596721827, State: Initialized, Role: FOLLOWER
I20260810 06:43:39.213462 13878 consensus_queue.cc:260] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } }
I20260810 06:43:39.214308 13878 ts_tablet_manager.cc:1428] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 06:43:39.214936 13877 tablet_bootstrap.cc:492] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: No bootstrap required, opened a new log
I20260810 06:43:39.215081 13877 ts_tablet_manager.cc:1397] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260810 06:43:39.215226 13879 tablet_bootstrap.cc:492] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: No bootstrap required, opened a new log
I20260810 06:43:39.215360 13879 ts_tablet_manager.cc:1397] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:43:39.215807 13877 raft_consensus.cc:348] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } }
I20260810 06:43:39.216029 13877 raft_consensus.cc:374] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:39.216127 13877 raft_consensus.cc:729] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e355e6966fc4b3dadbd4bb3e1a07424, State: Initialized, Role: FOLLOWER
I20260810 06:43:39.216094 13879 raft_consensus.cc:348] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } }
I20260810 06:43:39.216331 13879 raft_consensus.cc:374] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:39.216423 13879 raft_consensus.cc:729] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3abb014a3c14d20857c275f93d5ac40, State: Initialized, Role: FOLLOWER
I20260810 06:43:39.216336 13877 consensus_queue.cc:260] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } }
I20260810 06:43:39.216251 13886 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.216594 13886 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:39.216652 13879 consensus_queue.cc:260] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } }
I20260810 06:43:39.217167 13877 ts_tablet_manager.cc:1428] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:43:39.217824 13879 ts_tablet_manager.cc:1428] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:43:39.217782 13888 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.218173 13888 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:39.217018 13823 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.218444 13823 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:39.273765 13888 raft_consensus.cc:482] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:43:39.273970 13888 raft_consensus.cc:504] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } }
I20260810 06:43:39.275625 13888 leader_election.cc:288] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e355e6966fc4b3dadbd4bb3e1a07424 (127.10.242.1:40909), b92af91d8537459081c1a74596721827 (127.10.242.2:42611)
I20260810 06:43:39.284510 13895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.277148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:60451 (local address 127.10.242.2:42611)
0810 06:43:39.278097 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.278101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.278129 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:43:39.278218 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.278222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.278276 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.278350 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:39.278828 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.279851 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.282191 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.283382 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.283559 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.283626 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.284024 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.284173 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":712,"threads_started":1}
I20260810 06:43:39.284502 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.275932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:54873 (local address 127.10.242.1:40909)
0810 06:43:39.276493 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.276499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.276688 (+ 189us) server_negotiation.cc:407] Connection header received
0810 06:43:39.276959 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.276966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.277109 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.277242 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:39.278506 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.279619 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.282191 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.283379 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.283555 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.283654 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.284028 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.284166 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":256,"threads_started":1}
I20260810 06:43:39.284567 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.276785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:42611 (local address 127.10.242.3:60451)
0810 06:43:39.277921 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:39.277939 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:39.278044 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:39.278393 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:39.278397 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:39.278423 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:39.278675 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.278686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.279968 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.279973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.282008 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.282021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.283563 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.283569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.283838 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.283895 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 06:43:39.283980 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:43:39.284126 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"mutex_wait_us":110,"thread_start_us":811,"threads_started":1}
I20260810 06:43:39.284591 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.275882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:40909 (local address 127.10.242.3:54873)
0810 06:43:39.276573 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:39.276593 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:39.276737 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:39.278024 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:39.278028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:39.278052 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:39.278326 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.278338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.279773 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.279777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.282006 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.282021 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.283562 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.283569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.283843 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.283890 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 06:43:39.283978 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:43:39.284126 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":123,"thread_start_us":454,"threads_started":1}
I20260810 06:43:39.285275 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f5b74e0c579457f9a727d0f299f02d8" candidate_uuid: "c3abb014a3c14d20857c275f93d5ac40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92af91d8537459081c1a74596721827" is_pre_election: true
I20260810 06:43:39.285581 13475 raft_consensus.cc:2454] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3abb014a3c14d20857c275f93d5ac40 in term 0.
I20260810 06:43:39.285511 13252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f5b74e0c579457f9a727d0f299f02d8" candidate_uuid: "c3abb014a3c14d20857c275f93d5ac40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" is_pre_election: true
I20260810 06:43:39.285950 13252 raft_consensus.cc:2454] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3abb014a3c14d20857c275f93d5ac40 in term 0.
I20260810 06:43:39.286062 13583 leader_election.cc:302] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b92af91d8537459081c1a74596721827, c3abb014a3c14d20857c275f93d5ac40; no voters:
I20260810 06:43:39.286499 13888 raft_consensus.cc:2781] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:43:39.286633 13888 raft_consensus.cc:482] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:43:39.286752 13888 raft_consensus.cc:3037] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:39.288757 13888 raft_consensus.cc:504] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } }
I20260810 06:43:39.289283 13888 leader_election.cc:288] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [CANDIDATE]: Term 1 election: Requested vote from peers 9e355e6966fc4b3dadbd4bb3e1a07424 (127.10.242.1:40909), b92af91d8537459081c1a74596721827 (127.10.242.2:42611)
I20260810 06:43:39.289772 13252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f5b74e0c579457f9a727d0f299f02d8" candidate_uuid: "c3abb014a3c14d20857c275f93d5ac40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424"
I20260810 06:43:39.289753 13475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f5b74e0c579457f9a727d0f299f02d8" candidate_uuid: "c3abb014a3c14d20857c275f93d5ac40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92af91d8537459081c1a74596721827"
I20260810 06:43:39.289955 13252 raft_consensus.cc:3037] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:39.289973 13475 raft_consensus.cc:3037] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:39.292814 13475 raft_consensus.cc:2454] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3abb014a3c14d20857c275f93d5ac40 in term 1.
I20260810 06:43:39.292876 13252 raft_consensus.cc:2454] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3abb014a3c14d20857c275f93d5ac40 in term 1.
I20260810 06:43:39.293213 13583 leader_election.cc:302] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e355e6966fc4b3dadbd4bb3e1a07424, c3abb014a3c14d20857c275f93d5ac40; no voters:
I20260810 06:43:39.293468 13888 raft_consensus.cc:2781] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:39.293911 13888 raft_consensus.cc:686] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [term 1 LEADER]: Becoming Leader. State: Replica: c3abb014a3c14d20857c275f93d5ac40, State: Running, Role: LEADER
I20260810 06:43:39.294417 13888 consensus_queue.cc:237] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } }
I20260810 06:43:39.296211 13897 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: TxnStatusTablet state changed. Reason: New leader c3abb014a3c14d20857c275f93d5ac40. Latest consensus state: current_term: 1 leader_uuid: "c3abb014a3c14d20857c275f93d5ac40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.296497 13897 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:43:39.297271 13899 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:43:39.297552 13899 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:43:39.297897 13012 catalog_manager.cc:5495] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3abb014a3c14d20857c275f93d5ac40 (127.10.242.3). New cstate: current_term: 1 leader_uuid: "c3abb014a3c14d20857c275f93d5ac40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } health_report { overall_health: UNKNOWN } } }
I20260810 06:43:39.331925 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.325751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:39285 (local address 127.0.0.1:54176)
0810 06:43:39.325999 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:39.326022 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:39.326191 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:39.326841 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:39.326847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:39.326860 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:43:39.327067 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.327077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.328303 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.328309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.329290 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.329303 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.330053 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.330059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.330277 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.330334 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:43:39.331385 (+ 1051us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:43:39.331406 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 06:43:39.331487 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:43:39.331631 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 06:43:39.332288 13906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.325730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.10.242.3:39285)
0810 06:43:39.326408 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.326415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.326438 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:43:39.326543 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.326549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.326635 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.326789 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:39.327223 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.328164 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.329424 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.329903 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.330022 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.330563 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:39.331291 (+ 728us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:39.331425 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.331524 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.331601 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":373,"threads_started":1}
I20260810 06:43:39.336966 13475 raft_consensus.cc:1261] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Refusing update from remote peer c3abb014a3c14d20857c275f93d5ac40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:39.337172 13252 raft_consensus.cc:1261] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 FOLLOWER]: Refusing update from remote peer c3abb014a3c14d20857c275f93d5ac40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:39.338102 13897 consensus_queue.cc:1035] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:39.338402 13888 consensus_queue.cc:1035] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:39.345440 13886 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: TxnStatusTablet state changed. Reason: New leader c3abb014a3c14d20857c275f93d5ac40. Latest consensus state: current_term: 1 leader_uuid: "c3abb014a3c14d20857c275f93d5ac40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.345767 13886 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:39.351361 13908 mvcc.cc:205] Tried to move back new op lower bound from 7316865922396762112 to 7316865922233475072. Current Snapshot: MvccSnapshot[applied={T|T < 7316865922396762112}]
I20260810 06:43:39.364054 13823 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: TxnStatusTablet state changed. Reason: New leader c3abb014a3c14d20857c275f93d5ac40. Latest consensus state: current_term: 1 leader_uuid: "c3abb014a3c14d20857c275f93d5ac40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.364342 13823 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:39.364704 13886 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3abb014a3c14d20857c275f93d5ac40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.364928 13886 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P b92af91d8537459081c1a74596721827: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:39.368276 13888 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3abb014a3c14d20857c275f93d5ac40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.368589 13888 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:43:39.369769 13888 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3abb014a3c14d20857c275f93d5ac40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.370002 13888 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P c3abb014a3c14d20857c275f93d5ac40: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:43:39.378741 13823 tablet_replica.cc:439] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3abb014a3c14d20857c275f93d5ac40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e355e6966fc4b3dadbd4bb3e1a07424" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 40909 } } peers { permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 } } peers { permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 } } }
I20260810 06:43:39.379009 13823 tablet_replica.cc:442] T 3f5b74e0c579457f9a727d0f299f02d8 P 9e355e6966fc4b3dadbd4bb3e1a07424: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:39.392095 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.386180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:39285 (local address 127.10.242.1:48021)
0810 06:43:39.386753 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:39.386768 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:39.386885 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:39.387197 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:39.387200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:39.387209 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:43:39.387484 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.387491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.388883 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.388886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.389911 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.389923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.390938 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.390942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.391074 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.391126 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:43:39.391517 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:43:39.391525 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:43:39.391586 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 06:43:39.391716 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":304,"threads_started":1}
I20260810 06:43:39.392161 13906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.386132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:48021 (local address 127.10.242.3:39285)
0810 06:43:39.386302 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.386306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.386853 (+ 547us) server_negotiation.cc:407] Connection header received
0810 06:43:39.386994 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.387000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.387099 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.387193 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:39.387604 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.388769 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.390038 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.390791 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.390911 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.391290 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:39.391414 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:39.391518 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.391761 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.391863 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 06:43:39.401489 13687 raft_consensus.cc:1261] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Refusing update from remote peer 9e355e6966fc4b3dadbd4bb3e1a07424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:39.401531 13475 raft_consensus.cc:1261] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Refusing update from remote peer 9e355e6966fc4b3dadbd4bb3e1a07424: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:39.402109 13837 consensus_queue.cc:1035] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3abb014a3c14d20857c275f93d5ac40" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 39285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:39.402499 13823 consensus_queue.cc:1035] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92af91d8537459081c1a74596721827" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 42611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:39.406479 13909 mvcc.cc:205] Tried to move back new op lower bound from 7316865922663747584 to 7316865921197355008. Current Snapshot: MvccSnapshot[applied={T|T < 7316865922663747584}]
I20260810 06:43:39.437438 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.431143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:49245 (local address 127.10.242.1:40909)
0810 06:43:39.431330 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.431337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.431547 (+ 210us) server_negotiation.cc:407] Connection header received
0810 06:43:39.431752 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.431757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.431825 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.431910 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:39.432589 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.433525 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.435666 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.436080 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.436173 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.436722 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:39.436855 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:39.436940 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.437077 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.437157 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:43:39.437680 13930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.430721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:40909 (local address 127.10.242.3:49245)
0810 06:43:39.431445 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:39.431466 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:39.431613 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:39.432063 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:39.432067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:39.432077 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:43:39.432417 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.432433 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.433662 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.433668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.435527 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:39.435545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.436250 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.436257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:39.436455 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.436532 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:43:39.436945 (+ 413us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:43:39.436958 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 06:43:39.437026 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:43:39.437161 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":1016,"threads_started":1}
I20260810 06:43:39.466104 13012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:39.466408 13012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:39.469220 13012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a16c4e50927241aab6a04ecaf7557941: Sending DeleteTablet for 3 replicas of tablet c14abd53c16a44968bc747f268a693f0
I20260810 06:43:39.469847 13218 tablet_service.cc:1514] Processing DeleteTablet for tablet c14abd53c16a44968bc747f268a693f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:39 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:42968
I20260810 06:43:39.470165 13653 tablet_service.cc:1514] Processing DeleteTablet for tablet c14abd53c16a44968bc747f268a693f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:39 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:54168
I20260810 06:43:39.470263 13442 tablet_service.cc:1514] Processing DeleteTablet for tablet c14abd53c16a44968bc747f268a693f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:39 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:35988
I20260810 06:43:39.482352 13936 tablet_replica.cc:330] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: stopping tablet replica
I20260810 06:43:39.482676 13936 raft_consensus.cc:2229] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:43:39.483140 13936 raft_consensus.cc:2258] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:39.484161 13938 tablet_replica.cc:330] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: stopping tablet replica
I20260810 06:43:39.484272 13936 ts_tablet_manager.cc:1905] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:39.485146 13938 raft_consensus.cc:2229] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:39.485558 13938 raft_consensus.cc:2258] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:39.488803 13939 tablet_replica.cc:330] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: stopping tablet replica
I20260810 06:43:39.489034 13939 raft_consensus.cc:2229] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:39.489317 13939 raft_consensus.cc:2258] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:39.489578 13938 ts_tablet_manager.cc:1905] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:39.490990 13939 ts_tablet_manager.cc:1905] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:39.495044 13936 ts_tablet_manager.cc:1918] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:43:39.496907 13938 ts_tablet_manager.cc:1918] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:43:39.497115 13938 log.cc:1196] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/c14abd53c16a44968bc747f268a693f0
I20260810 06:43:39.497718 13938 ts_tablet_manager.cc:1939] T c14abd53c16a44968bc747f268a693f0 P c3abb014a3c14d20857c275f93d5ac40: Deleting consensus metadata
I20260810 06:43:39.498021 13939 ts_tablet_manager.cc:1918] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:43:39.498297 13939 log.cc:1196] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/c14abd53c16a44968bc747f268a693f0
I20260810 06:43:39.498503 13936 log.cc:1196] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/c14abd53c16a44968bc747f268a693f0
I20260810 06:43:39.498893 13939 ts_tablet_manager.cc:1939] T c14abd53c16a44968bc747f268a693f0 P b92af91d8537459081c1a74596721827: Deleting consensus metadata
I20260810 06:43:39.499014 13936 ts_tablet_manager.cc:1939] T c14abd53c16a44968bc747f268a693f0 P 9e355e6966fc4b3dadbd4bb3e1a07424: Deleting consensus metadata
I20260810 06:43:39.499410 12995 catalog_manager.cc:4841] TS c3abb014a3c14d20857c275f93d5ac40 (127.10.242.3:39285): tablet c14abd53c16a44968bc747f268a693f0 (table test-table [id=2c107281d3504001a5954d9c542711f3]) successfully deleted
I20260810 06:43:39.499960 12995 catalog_manager.cc:4841] TS b92af91d8537459081c1a74596721827 (127.10.242.2:42611): tablet c14abd53c16a44968bc747f268a693f0 (table test-table [id=2c107281d3504001a5954d9c542711f3]) successfully deleted
Aug 10 06:43:39 dist-test-slave-zj8q krb5kdc[12947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344218, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:43:39.501016 12995 catalog_manager.cc:4841] TS 9e355e6966fc4b3dadbd4bb3e1a07424 (127.10.242.1:40909): tablet c14abd53c16a44968bc747f268a693f0 (table test-table [id=2c107281d3504001a5954d9c542711f3]) successfully deleted
I20260810 06:43:39.507475 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.481261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42984 (local address 127.10.242.1:40909)
0810 06:43:39.481444 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.481451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.484925 (+ 3474us) server_negotiation.cc:407] Connection header received
0810 06:43:39.485980 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.485986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.486092 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.486182 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:39.489876 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.491127 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.495269 (+ 4142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.495882 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.495975 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.502074 (+ 6099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:39.502087 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:39.502103 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:39.502131 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:39.504820 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.505613 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.505620 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.505623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.505747 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.506115 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.506118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.506120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.506504 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:39.506714 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.507052 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.507205 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 06:43:39.508193 13324 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42984
I20260810 06:43:39.523661 13870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.511199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.10.242.62:41351)
0810 06:43:39.511421 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.511426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.511500 (+ 74us) server_negotiation.cc:407] Connection header received
0810 06:43:39.511654 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.511658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.511726 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.511800 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:39.513307 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.514018 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.515720 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.516135 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.516226 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.518946 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:39.518951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:39.518954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:39.518983 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:39.521329 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.522031 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.522035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.522037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.522141 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.522493 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.522496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.522497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.522758 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:39.522922 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.523271 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.523424 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
W20260810 06:43:39.524232 13012 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37114
I20260810 06:43:39.525990 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13117
I20260810 06:43:39.538048 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.527143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42992 (local address 127.10.242.1:40909)
0810 06:43:39.527315 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.527320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.527386 (+ 66us) server_negotiation.cc:407] Connection header received
0810 06:43:39.527547 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.527552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.527647 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.527732 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:39.528769 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.529429 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.530701 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.531144 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.531244 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.533224 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:39.533230 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:39.533234 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:39.533262 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:39.535655 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.536226 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.536232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.536236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.536350 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.536662 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.536667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.536670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.536988 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:39.537256 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.537534 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.537752 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 06:43:39.538614 13324 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42992
W20260810 06:43:39.539083 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13117: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:39.539250 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13117
I20260810 06:43:39.575596 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13338
Aug 10 06:43:39 dist-test-slave-zj8q krb5kdc[12947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344218, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:43:39.590085 13895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.576405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.10.242.2:42611)
0810 06:43:39.576616 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.576622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.576642 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:43:39.576748 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.576754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.576825 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.576908 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:39.578083 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.578868 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.580083 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.580481 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.580589 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.584785 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:39.584798 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:39.584813 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:39.584850 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:39.587584 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.588347 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.588354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.588357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.588491 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.588860 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.588863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.588865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.589176 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:39.589385 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.589685 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.589849 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 06:43:39.590905 13538 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35994
W20260810 06:43:39.591588 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13338: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:39.591740 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13338
I20260810 06:43:39.621649 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13553
Aug 10 06:43:39 dist-test-slave-zj8q krb5kdc[12947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344218, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:43:39.638680 13906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:39.622454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54188 (local address 127.10.242.3:39285)
0810 06:43:39.622647 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:39.622653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:39.622675 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:43:39.622810 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:39.622815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:39.622931 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:39.623061 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:39.624208 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.625327 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.626565 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:39.627259 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:39.627396 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:39.631680 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:39.631694 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:39.631713 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:39.631762 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:39.635738 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.636353 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.636363 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.636366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.636553 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:39.636908 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:39.636913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:39.636917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:39.637431 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:39.637786 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:39.637974 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 06:43:39.638252 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 06:43:39.639446 13768 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54188
W20260810 06:43:39.640148 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13553: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:39.640319 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13553
I20260810 06:43:39.668424 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 12967
I20260810 06:43:39.669515 13068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:39.783655 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 12967
2026-08-10T06:43:39Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4911 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:39 dist-test-slave-zj8q krb5kdc[13951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:39 dist-test-slave-zj8q krb5kdc[13951](info): set up 2 sockets
Aug 10 06:43:39 dist-test-slave-zj8q krb5kdc[13951](info): commencing operation
krb5kdc: starting...
W20260810 06:43:40.989472 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:43:41 dist-test-slave-zj8q krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:43:41.181753 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34407
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:34407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:41.498417 13973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:41.498710 13973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:41.498847 13973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:41.505740 13973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:41.505846 13973 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:43:41.505915 13973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:41.505995 13973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:41.506062 13973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:41.506145 13973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:41.515486 13973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:34407
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:41.516673 13973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:41.517957 13973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:41.528965 13983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:41.529119 13985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:41.529161 13988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:41.530004 13973 server_base.cc:1034] running on GCE node
I20260810 06:43:41.530804 13973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:41.532083 13973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:41.533303 13973 hybrid_clock.cc:648] HybridClock initialized: now 1786344221533239 us; error 30 us; skew 500 ppm
I20260810 06:43:41.533618 13973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:41 dist-test-slave-zj8q krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344221, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:41.536564 13973 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:43:41.537928 13973 webserver.cc:460] Webserver started at http://127.10.242.62:45795/ using document root <none> and password file <none>
I20260810 06:43:41.538656 13973 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:41.538760 13973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:41.539054 13973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:41.541850 13973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "c51fa6d9f66d49d29489dd30b92c511a"
format_stamp: "Formatted at 2026-08-10 06:43:41 on dist-test-slave-zj8q"
server_key: "2de51fb32ed5a01d7b047d0580311cfd"
server_key_iv: "b8079478342c200a7c61c1d0438ad82c"
server_key_version: "encryptionkey@0"
I20260810 06:43:41.542640 13973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "c51fa6d9f66d49d29489dd30b92c511a"
format_stamp: "Formatted at 2026-08-10 06:43:41 on dist-test-slave-zj8q"
server_key: "2de51fb32ed5a01d7b047d0580311cfd"
server_key_iv: "b8079478342c200a7c61c1d0438ad82c"
server_key_version: "encryptionkey@0"
I20260810 06:43:41.547477 13973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:43:41.550714 13999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:41.551921 13973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:41.552078 13973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "c51fa6d9f66d49d29489dd30b92c511a"
format_stamp: "Formatted at 2026-08-10 06:43:41 on dist-test-slave-zj8q"
server_key: "2de51fb32ed5a01d7b047d0580311cfd"
server_key_iv: "b8079478342c200a7c61c1d0438ad82c"
server_key_version: "encryptionkey@0"
I20260810 06:43:41.552275 13973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:41.567988 13973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:41.571702 13973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:41.571942 13973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:41.592037 13973 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:34407
I20260810 06:43:41.592058 14110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:34407 every 8 connection(s)
I20260810 06:43:41.593326 13973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:41.597121 14111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:41.597606 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13973
I20260810 06:43:41.597826 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:43:41.598697 11208 external_mini_cluster.cc:1428] Setting key 07cf359904ff8a37512e572faa1b36d7
I20260810 06:43:41.609601 14111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a: Bootstrap starting.
Aug 10 06:43:41 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:41.613063 14111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:41.614275 14111 log.cc:824] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:41.617276 14111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a: No bootstrap required, opened a new log
I20260810 06:43:41.621062 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:41.600809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.10.242.62:34407)
0810 06:43:41.601596 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:41.601607 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:41.601650 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:43:41.603576 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:41.603598 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:41.603987 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:41.604348 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:41.605796 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:41.607149 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:41.608387 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:41.609029 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:41.609204 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:41.613513 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:41.613520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:41.613534 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:41.613571 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:41.616655 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:41.617516 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:41.617523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:41.617528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:41.617686 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:41.618113 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:41.618116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:41.618118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:41.618776 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:41.619083 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:41.619631 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 06:43:41.620007 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":350,"threads_started":1}
I20260810 06:43:41.625674 14111 raft_consensus.cc:348] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51fa6d9f66d49d29489dd30b92c511a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34407 } }
I20260810 06:43:41.626010 14111 raft_consensus.cc:374] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:41.626111 14111 raft_consensus.cc:729] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c51fa6d9f66d49d29489dd30b92c511a, State: Initialized, Role: FOLLOWER
I20260810 06:43:41.626677 14111 consensus_queue.cc:260] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51fa6d9f66d49d29489dd30b92c511a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34407 } }
I20260810 06:43:41.626894 14111 raft_consensus.cc:388] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:41.626998 14111 raft_consensus.cc:482] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:41.627166 14111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:41.629478 14111 raft_consensus.cc:504] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51fa6d9f66d49d29489dd30b92c511a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34407 } }
I20260810 06:43:41.629959 14111 leader_election.cc:302] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c51fa6d9f66d49d29489dd30b92c511a; no voters:
I20260810 06:43:41.630533 14111 leader_election.cc:288] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:41.630657 14118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:41.631549 14118 raft_consensus.cc:686] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [term 1 LEADER]: Becoming Leader. State: Replica: c51fa6d9f66d49d29489dd30b92c511a, State: Running, Role: LEADER
I20260810 06:43:41.632035 14118 consensus_queue.cc:237] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51fa6d9f66d49d29489dd30b92c511a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34407 } }
I20260810 06:43:41.632400 14111 sys_catalog.cc:564] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:41.634595 14121 sys_catalog.cc:455] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c51fa6d9f66d49d29489dd30b92c511a. Latest consensus state: current_term: 1 leader_uuid: "c51fa6d9f66d49d29489dd30b92c511a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51fa6d9f66d49d29489dd30b92c511a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34407 } } }
I20260810 06:43:41.635195 14121 sys_catalog.cc:458] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:41.635666 14120 sys_catalog.cc:455] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c51fa6d9f66d49d29489dd30b92c511a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c51fa6d9f66d49d29489dd30b92c511a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34407 } } }
I20260810 06:43:41.636057 14120 sys_catalog.cc:458] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:41.636382 14129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:41.642186 14129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:41.657392 14129 catalog_manager.cc:1347] Generated new cluster ID: ec20ffd2f63a499faba8c48e4f1242df
I20260810 06:43:41.657522 14129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:41.666033 14129 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:41.667339 14129 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:41.678891 14129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a: Generated new TSK 0
I20260810 06:43:41.679715 14129 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:43:41.736223 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34407
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:42.064707 14152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:42.065002 14152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:42.065150 14152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:42.072477 14152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:42.072606 14152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:42.072736 14152 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:43:42.072805 14152 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:43:42.072911 14152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:43:42.082614 14152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.62:34407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:42.083964 14152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:42.085342 14152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:42.098270 14164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:42.098414 14161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:42.098308 14162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:42.099368 14152 server_base.cc:1034] running on GCE node
I20260810 06:43:42.099932 14152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:42.100829 14152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:42.102048 14152 hybrid_clock.cc:648] HybridClock initialized: now 1786344222101977 us; error 55 us; skew 500 ppm
I20260810 06:43:42.102402 14152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:42 dist-test-slave-zj8q krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344222, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:42.105610 14152 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:43:42.107095 14152 webserver.cc:460] Webserver started at http://127.10.242.1:41529/ using document root <none> and password file <none>
I20260810 06:43:42.107748 14152 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:42.107861 14152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:42.108183 14152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:42.111358 14152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "9c18629ed3134db7b12e2ef43873ecd5"
format_stamp: "Formatted at 2026-08-10 06:43:42 on dist-test-slave-zj8q"
server_key: "9f4240d91f5790a6fb3b77b5266361cd"
server_key_iv: "f1b8557d2531d4817518b0f671f15d83"
server_key_version: "encryptionkey@0"
I20260810 06:43:42.112226 14152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "9c18629ed3134db7b12e2ef43873ecd5"
format_stamp: "Formatted at 2026-08-10 06:43:42 on dist-test-slave-zj8q"
server_key: "9f4240d91f5790a6fb3b77b5266361cd"
server_key_iv: "f1b8557d2531d4817518b0f671f15d83"
server_key_version: "encryptionkey@0"
I20260810 06:43:42.117386 14152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:43:42.120930 14175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:42.122414 14152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:43:42.122575 14152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "9c18629ed3134db7b12e2ef43873ecd5"
format_stamp: "Formatted at 2026-08-10 06:43:42 on dist-test-slave-zj8q"
server_key: "9f4240d91f5790a6fb3b77b5266361cd"
server_key_iv: "f1b8557d2531d4817518b0f671f15d83"
server_key_version: "encryptionkey@0"
I20260810 06:43:42.122804 14152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:42.163899 14152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:42.168361 14152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:42.168637 14152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:42.171312 14152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:42.171419 14152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:42.171557 14152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:42.171692 14152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 10 06:43:42 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344222, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:42.194365 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:42.171765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:51937 (local address 127.10.242.62:34407)
0810 06:43:42.172582 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:42.172590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:42.173518 (+ 928us) server_negotiation.cc:407] Connection header received
0810 06:43:42.176321 (+ 2803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:42.176328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:42.176419 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:42.176547 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:42.178963 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.179837 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.181823 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.182289 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.182397 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:42.187056 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:42.187063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:42.187069 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:42.187115 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:42.190523 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:42.191522 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:42.191528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:42.191532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:42.191695 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:42.192340 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:42.192346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:42.192349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:42.192647 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:42.192794 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:42.193746 (+ 952us) server_negotiation.cc:299] Negotiation successful
0810 06:43:42.193976 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":580,"threads_started":1}
I20260810 06:43:42.195454 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:42.172409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34407 (local address 127.10.242.1:51937)
0810 06:43:42.173338 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:42.173390 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:42.175977 (+ 2587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:42.176841 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:42.176854 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:42.177704 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:42.178710 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:42.178743 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.180012 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.180019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:42.181679 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:42.181694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.182553 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.182561 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:42.182803 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:42.183532 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:42.183559 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:42.186794 (+ 3235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:42.190709 (+ 3915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:42.190718 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:42.190739 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:42.191392 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:42.191848 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:42.191853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:42.191857 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:42.192195 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:42.192880 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:42.192889 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:42.193338 (+ 449us) client_negotiation.cc:769] Sending connection context
0810 06:43:42.193599 (+ 261us) client_negotiation.cc:240] Negotiation successful
0810 06:43:42.194214 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":370,"threads_started":1}
I20260810 06:43:42.215888 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:42.201514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:40489 (local address 127.10.242.62:34407)
0810 06:43:42.201711 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:42.201716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:42.202714 (+ 998us) server_negotiation.cc:407] Connection header received
0810 06:43:42.202846 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:42.202852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:42.202945 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:42.203051 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:42.204032 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.205093 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.206555 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.207185 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.207313 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:42.209396 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:42.209402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:42.209407 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:42.209446 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:42.212836 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:42.213735 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:42.213741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:42.213745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:42.213903 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:42.214362 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:42.214366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:42.214370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:42.214739 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:42.214872 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:42.215279 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 06:43:42.215511 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 06:43:42.215951 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:42.201648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34407 (local address 127.10.242.1:40489)
0810 06:43:42.202615 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:42.202633 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:42.202749 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:42.203102 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:42.203108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:42.203624 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:42.203901 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:42.203910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.205245 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.205249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:42.206393 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:42.206414 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.207383 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.207389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:42.207579 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:42.208229 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:42.208257 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:42.209200 (+ 943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:42.212969 (+ 3769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:42.212975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:42.212979 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:42.213590 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:42.214032 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:42.214038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:42.214045 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:42.214246 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:42.214862 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:42.214865 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:42.215017 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 06:43:42.215130 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 06:43:42.215428 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"thread_start_us":684,"threads_started":1}
I20260810 06:43:42.216727 14152 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:43535
I20260810 06:43:42.217399 14364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:43535 every 8 connection(s)
I20260810 06:43:42.218083 14152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:43:42.218760 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 14152
I20260810 06:43:42.218976 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:43:42.219779 11208 external_mini_cluster.cc:1428] Setting key b5686af3357dba8cd1115d9f0c494be7
I20260810 06:43:42.220708 14366 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34407
I20260810 06:43:42.220928 14366 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:42.221657 14366 heartbeater.cc:511] Master 127.10.242.62:34407 requested a full tablet report, sending...
I20260810 06:43:42.223986 14032 ts_manager.cc:194] Registered new tserver with Master: 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535)
I20260810 06:43:42.225842 14032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:51937
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:43:42.265161 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34407
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:42.580209 14373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:42.580469 14373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:42.580578 14373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:42.587514 14373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:42.587625 14373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:42.587718 14373 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:43:42.587767 14373 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:43:42.587834 14373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:43:42.597180 14373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.62:34407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:42.598511 14373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:42.599779 14373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:42.612140 14383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:42.612170 14384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:42.612355 14386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:42.612910 14373 server_base.cc:1034] running on GCE node
I20260810 06:43:42.613523 14373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:42.614390 14373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:42.615579 14373 hybrid_clock.cc:648] HybridClock initialized: now 1786344222615536 us; error 32 us; skew 500 ppm
I20260810 06:43:42.615863 14373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:42 dist-test-slave-zj8q krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344222, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:42.618824 14373 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:43:42.620234 14373 webserver.cc:460] Webserver started at http://127.10.242.2:39545/ using document root <none> and password file <none>
I20260810 06:43:42.620898 14373 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:42.621016 14373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:42.621315 14373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:42.624142 14373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "220f89114a334ea2ab42a24dece98c54"
format_stamp: "Formatted at 2026-08-10 06:43:42 on dist-test-slave-zj8q"
server_key: "6cd553ba352b4ca37699690395047162"
server_key_iv: "679677b4b026c45223c19b9df2b535b3"
server_key_version: "encryptionkey@0"
I20260810 06:43:42.624882 14373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "220f89114a334ea2ab42a24dece98c54"
format_stamp: "Formatted at 2026-08-10 06:43:42 on dist-test-slave-zj8q"
server_key: "6cd553ba352b4ca37699690395047162"
server_key_iv: "679677b4b026c45223c19b9df2b535b3"
server_key_version: "encryptionkey@0"
I20260810 06:43:42.629977 14373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:43:42.633285 14394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:42.634605 14373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:42.634770 14373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "220f89114a334ea2ab42a24dece98c54"
format_stamp: "Formatted at 2026-08-10 06:43:42 on dist-test-slave-zj8q"
server_key: "6cd553ba352b4ca37699690395047162"
server_key_iv: "679677b4b026c45223c19b9df2b535b3"
server_key_version: "encryptionkey@0"
I20260810 06:43:42.634948 14373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:42.647456 14373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:42.651516 14373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:42.651791 14373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:42.654337 14373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:42.654459 14373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:42.654572 14373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:42.654662 14373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:43:42 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344222, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:42.676646 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:42.655017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:56517 (local address 127.10.242.62:34407)
0810 06:43:42.655227 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:42.655235 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:42.656604 (+ 1369us) server_negotiation.cc:407] Connection header received
0810 06:43:42.658794 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:42.658801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:42.658881 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:42.658966 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:42.660421 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.661469 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.664425 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.665002 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.665145 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:42.669751 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:42.669758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:42.669764 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:42.669801 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:42.672908 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:42.673858 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:42.673865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:42.673870 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:42.674039 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:42.674551 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:42.674557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:42.674561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:42.674863 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:42.674999 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:42.675969 (+ 970us) server_negotiation.cc:299] Negotiation successful
0810 06:43:42.676244 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 06:43:42.677614 14407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:42.655621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34407 (local address 127.10.242.2:56517)
0810 06:43:42.656422 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:42.656492 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:42.658467 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:42.659227 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:42.659234 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:42.659705 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:42.660245 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:42.660263 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.661666 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.661673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:42.664250 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:42.664270 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.665411 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.665416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:42.665612 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:42.666293 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:42.666323 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:42.669508 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:42.673049 (+ 3541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:42.673058 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:42.673075 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:42.673703 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:42.674175 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:42.674180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:42.674183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:42.674421 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:42.674992 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:42.674999 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:42.675474 (+ 475us) client_negotiation.cc:769] Sending connection context
0810 06:43:42.675810 (+ 336us) client_negotiation.cc:240] Negotiation successful
0810 06:43:42.676226 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":327,"threads_started":1}
I20260810 06:43:42.697077 14373 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:46529
I20260810 06:43:42.697118 14584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:46529 every 8 connection(s)
I20260810 06:43:42.697763 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:42.683485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:34791 (local address 127.10.242.62:34407)
0810 06:43:42.683647 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:42.683652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:42.684539 (+ 887us) server_negotiation.cc:407] Connection header received
0810 06:43:42.684705 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:42.684711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:42.684778 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:42.684854 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:42.686249 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.687152 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.688954 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.689356 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.689463 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:42.691512 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:42.691519 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:42.691523 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:42.691553 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:42.694710 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:42.695481 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:42.695487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:42.695491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:42.695636 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:42.696058 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:42.696063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:42.696066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:42.696469 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:42.696621 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:42.697187 (+ 566us) server_negotiation.cc:299] Negotiation successful
0810 06:43:42.697396 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:43:42.697868 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:42.683727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34407 (local address 127.10.242.2:34791)
0810 06:43:42.684444 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:42.684460 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:42.684593 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:42.684908 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:42.684913 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:42.685603 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:42.686020 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:42.686038 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.687345 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.687349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:42.688809 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:42.688825 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:42.689538 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:42.689544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:42.689685 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:42.690315 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:42.690332 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:42.691328 (+ 996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:42.694847 (+ 3519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:42.694856 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:42.694860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:42.695369 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:42.695748 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:42.695753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:42.695757 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:42.695915 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:42.696710 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:42.696715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:42.696923 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 06:43:42.697075 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 06:43:42.697426 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":389,"threads_started":1}
I20260810 06:43:42.698848 14373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:43:42.700978 14585 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34407
I20260810 06:43:42.701212 14585 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:42.701836 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 14373
I20260810 06:43:42.701931 14585 heartbeater.cc:511] Master 127.10.242.62:34407 requested a full tablet report, sending...
I20260810 06:43:42.702019 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:43:42.702502 11208 external_mini_cluster.cc:1428] Setting key 46ff79901f0166895cb34329bf2e5b48
I20260810 06:43:42.703227 14032 ts_manager.cc:194] Registered new tserver with Master: 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529)
I20260810 06:43:42.704078 14032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:56517
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:43:42.744968 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34407
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:43.057799 14591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:43.058068 14591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:43.058238 14591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:43.065146 14591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:43.065259 14591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:43.065382 14591 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:43:43.065448 14591 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:43:43.065563 14591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:43:43.074681 14591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.62:34407
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:43.075860 14591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:43.077220 14591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:43.089691 14596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:43.089763 14599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:43.089779 14597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:43.090787 14591 server_base.cc:1034] running on GCE node
I20260810 06:43:43.091361 14591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:43.092206 14591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:43.093403 14591 hybrid_clock.cc:648] HybridClock initialized: now 1786344223093355 us; error 39 us; skew 500 ppm
I20260810 06:43:43.093689 14591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:43 dist-test-slave-zj8q krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344223, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:43.099637 14591 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:43:43.101024 14591 webserver.cc:460] Webserver started at http://127.10.242.3:34779/ using document root <none> and password file <none>
I20260810 06:43:43.101655 14591 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:43.101771 14591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:43.102068 14591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:43.104856 14591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "72ae479d89ed4f7db0d5b888874f8ac6"
format_stamp: "Formatted at 2026-08-10 06:43:43 on dist-test-slave-zj8q"
server_key: "5a3be408e36721b79914fbed6a4c4c62"
server_key_iv: "f1f9752e4c5b491f682956795c4cad38"
server_key_version: "encryptionkey@0"
I20260810 06:43:43.105592 14591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "72ae479d89ed4f7db0d5b888874f8ac6"
format_stamp: "Formatted at 2026-08-10 06:43:43 on dist-test-slave-zj8q"
server_key: "5a3be408e36721b79914fbed6a4c4c62"
server_key_iv: "f1f9752e4c5b491f682956795c4cad38"
server_key_version: "encryptionkey@0"
I20260810 06:43:43.110420 14591 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260810 06:43:43.113528 14608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:43.114771 14591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:43:43.114918 14591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "72ae479d89ed4f7db0d5b888874f8ac6"
format_stamp: "Formatted at 2026-08-10 06:43:43 on dist-test-slave-zj8q"
server_key: "5a3be408e36721b79914fbed6a4c4c62"
server_key_iv: "f1f9752e4c5b491f682956795c4cad38"
server_key_version: "encryptionkey@0"
I20260810 06:43:43.115084 14591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:43.148164 14591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:43.152449 14591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:43.152733 14591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:43.155512 14591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:43.155645 14591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:43.155776 14591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:43.155870 14591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:43:43 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344223, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:43.179785 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.156055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:59955 (local address 127.10.242.62:34407)
0810 06:43:43.156282 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.156288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.157796 (+ 1508us) server_negotiation.cc:407] Connection header received
0810 06:43:43.159875 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.159881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.159971 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.160081 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:43.162865 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.163768 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.166638 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.167109 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.167246 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.172192 (+ 4946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:43.172197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:43.172200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:43.172234 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:43.175786 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:43.176722 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:43.176727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:43.176729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:43.176892 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:43.177536 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:43.177542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:43.177545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:43.177897 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:43.178048 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.179118 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.179376 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 06:43:43.181454 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.156791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34407 (local address 127.10.242.3:59955)
0810 06:43:43.157627 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.157668 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.159581 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.160411 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.160427 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.161426 (+ 999us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:43.162550 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.162585 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.163944 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.163953 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.166466 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.166485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.167391 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.167398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.167688 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.168476 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:43.168510 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:43.171930 (+ 3420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:43.175950 (+ 4020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:43.175961 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:43.175972 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:43.176588 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:43.177037 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:43.177043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:43.177047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:43.177378 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:43.178022 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:43.178042 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:43.178566 (+ 524us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.178945 (+ 379us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.179597 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":379,"threads_started":1}
I20260810 06:43:43.201081 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.187438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:35657 (local address 127.10.242.62:34407)
0810 06:43:43.187587 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.187591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.188209 (+ 618us) server_negotiation.cc:407] Connection header received
0810 06:43:43.188362 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.188367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.188438 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.188518 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:43.189410 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.190090 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.192242 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.192893 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.193050 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.195100 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:43.195106 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:43.195111 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:43.195153 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:43.198311 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:43.199123 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:43.199129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:43.199133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:43.199266 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:43.199615 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:43.199620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:43.199624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:43.199901 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:43.200055 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.200451 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.200719 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:43:43.201143 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.187525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34407 (local address 127.10.242.3:35657)
0810 06:43:43.188096 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.188111 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.188252 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.188558 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.188562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.188999 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:43.189275 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.189285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.190266 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.190271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.192011 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.192025 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.193094 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.193102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.193274 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.193846 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:43.193863 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:43.194875 (+ 1012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:43.198433 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:43.198439 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:43.198443 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:43.198989 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:43.199373 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:43.199379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:43.199383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:43.199511 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:43.200037 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:43.200042 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:43.200219 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.200337 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.200689 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":311,"threads_started":1}
I20260810 06:43:43.208830 14591 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:42159
I20260810 06:43:43.208861 14819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:42159 every 8 connection(s)
I20260810 06:43:43.209991 14591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:43:43.212054 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 14591
I20260810 06:43:43.212141 14820 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34407
I20260810 06:43:43.212236 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:43:43.212373 14820 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:43.212725 11208 external_mini_cluster.cc:1428] Setting key 7011ce22c94d0b9db33ed1c740666648
I20260810 06:43:43.213029 14820 heartbeater.cc:511] Master 127.10.242.62:34407 requested a full tablet report, sending...
I20260810 06:43:43.214696 14032 ts_manager.cc:194] Registered new tserver with Master: 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159)
I20260810 06:43:43.215912 14032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:59955
I20260810 06:43:43.216012 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:43:43.228700 14366 heartbeater.cc:503] Master 127.10.242.62:34407 was elected leader, sending a full tablet report...
Aug 10 06:43:43 dist-test-slave-zj8q krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:43:43 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:43.267303 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.254418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.10.242.62:34407)
0810 06:43:43.254646 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.254652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.254935 (+ 283us) server_negotiation.cc:407] Connection header received
0810 06:43:43.255082 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.255085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.255170 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.255250 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:43.256638 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.257380 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.258579 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.258996 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.259086 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.262525 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:43.262529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:43.262532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:43.262570 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:43.264810 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:43.265599 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:43.265604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:43.265609 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:43.265782 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:43.266121 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:43.266124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:43.266126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:43.266442 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:43.266639 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.266857 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.267024 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:43:43.270932 14032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:43:43.277532 14032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:43:43.299597 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.291101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.10.242.2:46529)
0810 06:43:43.291943 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.291950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.291977 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:43:43.292067 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.292074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.292317 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.293259 (+ 942us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:43.293424 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.294584 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.297357 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.298488 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.298617 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.298878 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.299151 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.299321 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":582,"threads_started":1}
I20260810 06:43:43.299861 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.290853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:46529 (local address 127.0.0.1:49614)
0810 06:43:43.291366 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.291386 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.291489 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.292541 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.292546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.292573 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:43.293030 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.293046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.294716 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.294721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.297160 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.297173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.298706 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.298714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.298901 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.298953 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.299093 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.299305 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":22,"thread_start_us":254,"threads_started":1}
I20260810 06:43:43.300542 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.289412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56792 (local address 127.10.242.3:42159)
0810 06:43:43.290282 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.290291 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.290652 (+ 361us) server_negotiation.cc:407] Connection header received
0810 06:43:43.290926 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.290934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.291182 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.291404 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:43.293305 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.294847 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.297368 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.298576 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.298698 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.298933 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.300054 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.300227 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":412,"threads_started":1}
I20260810 06:43:43.300662 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.291426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:43535 (local address 127.0.0.1:52416)
0810 06:43:43.292271 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.292293 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.292400 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.293456 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.293460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.293485 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:43.293836 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.293848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.295696 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.295701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.297761 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.297776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.299939 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.299947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.300110 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.300143 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.300206 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.300317 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":3,"thread_start_us":683,"threads_started":1}
I20260810 06:43:43.299919 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.289154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42159 (local address 127.0.0.1:56792)
0810 06:43:43.290513 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.290555 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.290693 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.291750 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.291757 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.291795 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:43.293048 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.293061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.294983 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.294988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.297158 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.297173 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.299044 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.299051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.299224 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.299259 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.299349 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.299483 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1159,"mutex_wait_us":6,"thread_start_us":1039,"threads_started":1}
I20260810 06:43:43.301734 14853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.291935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.10.242.1:43535)
0810 06:43:43.292897 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.292905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.292933 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:43:43.293036 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.293043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.293262 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.293463 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:43.294012 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.295555 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.297929 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.299689 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.301037 (+ 1348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.301230 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.301306 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.301457 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":655,"threads_started":1}
I20260810 06:43:43.304013 14696 tablet_service.cc:1467] Processing CreateTablet for tablet 73b822f4d990406483a0d90649c820a3 (DEFAULT_TABLE table=test-table [id=a041dfebf6a445ffbbe86c1c64308993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:43.303818 14473 tablet_service.cc:1467] Processing CreateTablet for tablet 73b822f4d990406483a0d90649c820a3 (DEFAULT_TABLE table=test-table [id=a041dfebf6a445ffbbe86c1c64308993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:43.305294 14473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b822f4d990406483a0d90649c820a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:43.305295 14696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b822f4d990406483a0d90649c820a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:43.305081 14258 tablet_service.cc:1467] Processing CreateTablet for tablet 73b822f4d990406483a0d90649c820a3 (DEFAULT_TABLE table=test-table [id=a041dfebf6a445ffbbe86c1c64308993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:43.306401 14258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b822f4d990406483a0d90649c820a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:43.316133 14854 tablet_bootstrap.cc:492] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: Bootstrap starting.
I20260810 06:43:43.320075 14858 tablet_bootstrap.cc:492] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: Bootstrap starting.
I20260810 06:43:43.320255 14854 tablet_bootstrap.cc:654] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:43.321612 14854 log.cc:824] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:43.323552 14858 tablet_bootstrap.cc:654] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:43.324816 14854 tablet_bootstrap.cc:492] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: No bootstrap required, opened a new log
I20260810 06:43:43.324790 14858 log.cc:824] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:43.325084 14854 ts_tablet_manager.cc:1397] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:43:43.326248 14856 tablet_bootstrap.cc:492] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: Bootstrap starting.
I20260810 06:43:43.327715 14858 tablet_bootstrap.cc:492] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: No bootstrap required, opened a new log
I20260810 06:43:43.328053 14858 ts_tablet_manager.cc:1397] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 06:43:43.329664 14856 tablet_bootstrap.cc:654] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:43.330627 14856 log.cc:824] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:43.332962 14856 tablet_bootstrap.cc:492] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: No bootstrap required, opened a new log
I20260810 06:43:43.333194 14856 ts_tablet_manager.cc:1397] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260810 06:43:43.335039 14854 raft_consensus.cc:348] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } }
I20260810 06:43:43.335454 14854 raft_consensus.cc:374] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:43.335546 14854 raft_consensus.cc:729] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ae479d89ed4f7db0d5b888874f8ac6, State: Initialized, Role: FOLLOWER
I20260810 06:43:43.336138 14854 consensus_queue.cc:260] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } }
I20260810 06:43:43.337215 14820 heartbeater.cc:503] Master 127.10.242.62:34407 was elected leader, sending a full tablet report...
I20260810 06:43:43.337972 14858 raft_consensus.cc:348] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } }
I20260810 06:43:43.338510 14858 raft_consensus.cc:374] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:43.338680 14858 raft_consensus.cc:729] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c18629ed3134db7b12e2ef43873ecd5, State: Initialized, Role: FOLLOWER
I20260810 06:43:43.339501 14854 ts_tablet_manager.cc:1428] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 06:43:43.339460 14858 consensus_queue.cc:260] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } }
I20260810 06:43:43.341094 14858 ts_tablet_manager.cc:1428] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 06:43:43.341933 14856 raft_consensus.cc:348] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } }
I20260810 06:43:43.342362 14856 raft_consensus.cc:374] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:43.342471 14856 raft_consensus.cc:729] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220f89114a334ea2ab42a24dece98c54, State: Initialized, Role: FOLLOWER
I20260810 06:43:43.343065 14856 consensus_queue.cc:260] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } }
I20260810 06:43:43.344220 14585 heartbeater.cc:503] Master 127.10.242.62:34407 was elected leader, sending a full tablet report...
I20260810 06:43:43.344738 14856 ts_tablet_manager.cc:1428] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260810 06:43:43.404553 14866 raft_consensus.cc:482] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:43:43.404860 14866 raft_consensus.cc:504] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } }
I20260810 06:43:43.406616 14866 leader_election.cc:288] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159), 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529)
I20260810 06:43:43.414224 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.407961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:53567 (local address 127.10.242.2:46529)
0810 06:43:43.408427 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.408433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.408502 (+ 69us) server_negotiation.cc:407] Connection header received
0810 06:43:43.408662 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.408667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.408740 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.408832 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:43.409412 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.410295 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.412528 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.413271 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.413391 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.413459 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.413800 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.413931 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323}
I20260810 06:43:43.414233 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.406986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:53823 (local address 127.10.242.3:42159)
0810 06:43:43.407191 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.407198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.407768 (+ 570us) server_negotiation.cc:407] Connection header received
0810 06:43:43.407992 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.407999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.408087 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.408181 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:43.408951 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.409970 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.412599 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.413342 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.413509 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.413580 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.413860 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.413985 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:43:43.414441 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.407817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:46529 (local address 127.10.242.1:53567)
0810 06:43:43.408417 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.408430 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.408545 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.408857 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.408860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.408881 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:43.409247 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.409260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.410423 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.410428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.412425 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.412435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.413472 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.413479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.413649 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.413672 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.413746 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.413850 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":25,"thread_start_us":262,"threads_started":1}
I20260810 06:43:43.414496 14871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.406987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42159 (local address 127.10.242.1:53823)
0810 06:43:43.407660 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.407677 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.407825 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.408377 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.408381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.408414 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:43.408797 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.408812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.410127 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.410130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.412398 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.412413 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.413576 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.413580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.413694 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.413727 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.413821 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.413940 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":3,"thread_start_us":394,"threads_started":1}
I20260810 06:43:43.415408 14501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b822f4d990406483a0d90649c820a3" candidate_uuid: "9c18629ed3134db7b12e2ef43873ecd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220f89114a334ea2ab42a24dece98c54" is_pre_election: true
I20260810 06:43:43.415407 14729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b822f4d990406483a0d90649c820a3" candidate_uuid: "9c18629ed3134db7b12e2ef43873ecd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" is_pre_election: true
I20260810 06:43:43.415875 14729 raft_consensus.cc:2454] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c18629ed3134db7b12e2ef43873ecd5 in term 0.
I20260810 06:43:43.415880 14501 raft_consensus.cc:2454] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c18629ed3134db7b12e2ef43873ecd5 in term 0.
I20260810 06:43:43.416477 14180 leader_election.cc:302] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ae479d89ed4f7db0d5b888874f8ac6, 9c18629ed3134db7b12e2ef43873ecd5; no voters:
I20260810 06:43:43.416819 14866 raft_consensus.cc:2781] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:43:43.416951 14866 raft_consensus.cc:482] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:43:43.417047 14866 raft_consensus.cc:3037] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:43.419328 14866 raft_consensus.cc:504] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } }
I20260810 06:43:43.419854 14866 leader_election.cc:288] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [CANDIDATE]: Term 1 election: Requested vote from peers 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159), 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529)
I20260810 06:43:43.420303 14729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b822f4d990406483a0d90649c820a3" candidate_uuid: "9c18629ed3134db7b12e2ef43873ecd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ae479d89ed4f7db0d5b888874f8ac6"
I20260810 06:43:43.420331 14501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b822f4d990406483a0d90649c820a3" candidate_uuid: "9c18629ed3134db7b12e2ef43873ecd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220f89114a334ea2ab42a24dece98c54"
I20260810 06:43:43.420466 14729 raft_consensus.cc:3037] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:43.420495 14501 raft_consensus.cc:3037] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:43.422691 14729 raft_consensus.cc:2454] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c18629ed3134db7b12e2ef43873ecd5 in term 1.
I20260810 06:43:43.422734 14501 raft_consensus.cc:2454] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c18629ed3134db7b12e2ef43873ecd5 in term 1.
I20260810 06:43:43.423056 14180 leader_election.cc:302] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ae479d89ed4f7db0d5b888874f8ac6, 9c18629ed3134db7b12e2ef43873ecd5; no voters:
I20260810 06:43:43.423305 14866 raft_consensus.cc:2781] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:43.423700 14866 raft_consensus.cc:686] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 LEADER]: Becoming Leader. State: Replica: 9c18629ed3134db7b12e2ef43873ecd5, State: Running, Role: LEADER
I20260810 06:43:43.424214 14866 consensus_queue.cc:237] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } }
I20260810 06:43:43.428799 14029 catalog_manager.cc:5495] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "9c18629ed3134db7b12e2ef43873ecd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } health_report { overall_health: HEALTHY } } }
W20260810 06:43:43.450861 14586 tablet.cc:2369] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:43:43.461535 14822 tablet.cc:2369] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:43:43 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344221, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
W20260810 06:43:43.473726 14367 tablet.cc:2369] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:43:43.476966 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.464385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44560 (local address 127.10.242.62:34407)
0810 06:43:43.464548 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.464554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.465104 (+ 550us) server_negotiation.cc:407] Connection header received
0810 06:43:43.465265 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.465268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.465335 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.465413 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:43.466575 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.467324 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.468547 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.468969 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.469075 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.472136 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:43.472140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:43.472144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:43.472173 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:43.474405 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:43.475056 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:43.475061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:43.475063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:43.475184 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:43.475523 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:43.475528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:43.475531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:43.475804 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:43.475931 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.476340 (+ 409us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.476624 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":52}
I20260810 06:43:43.477000 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.464402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34407 (local address 127.0.0.1:44560)
0810 06:43:43.465006 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.465020 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.465154 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.465456 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.465460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.466134 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:43.466442 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.466452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.467444 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.467447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.468421 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.468433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.469140 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.469146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.469298 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.469846 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:43.469863 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:43.471976 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:43.474521 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:43.474533 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:43.474542 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:43.474949 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:43.475280 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:43.475285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:43.475289 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:43.475427 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:43.475931 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:43.475939 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:43.476085 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.476234 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.476487 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":335,"threads_started":1}
I20260810 06:43:43.481714 14029 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:44560:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:43:43.482779 14029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:43:43.489307 14473 tablet_service.cc:1467] Processing CreateTablet for tablet 10ea54cd95c44d4e8663068e8e315cbd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f12bad2e7802464595c57d7be4a0bcb5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:43:43.489085 14258 tablet_service.cc:1467] Processing CreateTablet for tablet 10ea54cd95c44d4e8663068e8e315cbd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f12bad2e7802464595c57d7be4a0bcb5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:43:43.489465 14696 tablet_service.cc:1467] Processing CreateTablet for tablet 10ea54cd95c44d4e8663068e8e315cbd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f12bad2e7802464595c57d7be4a0bcb5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:43:43.490095 14258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ea54cd95c44d4e8663068e8e315cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:43.490092 14473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ea54cd95c44d4e8663068e8e315cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:43.490283 14696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ea54cd95c44d4e8663068e8e315cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:43.495704 14854 tablet_bootstrap.cc:492] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: Bootstrap starting.
I20260810 06:43:43.497138 14854 tablet_bootstrap.cc:654] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:43.497251 14858 tablet_bootstrap.cc:492] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: Bootstrap starting.
I20260810 06:43:43.497753 14856 tablet_bootstrap.cc:492] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: Bootstrap starting.
I20260810 06:43:43.498904 14858 tablet_bootstrap.cc:654] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:43.499409 14856 tablet_bootstrap.cc:654] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:43.499445 14854 tablet_bootstrap.cc:492] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: No bootstrap required, opened a new log
I20260810 06:43:43.499583 14854 ts_tablet_manager.cc:1397] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260810 06:43:43.500387 14854 raft_consensus.cc:348] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.500638 14854 raft_consensus.cc:374] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:43.500741 14854 raft_consensus.cc:729] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ae479d89ed4f7db0d5b888874f8ac6, State: Initialized, Role: FOLLOWER
I20260810 06:43:43.500988 14854 consensus_queue.cc:260] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.501188 14858 tablet_bootstrap.cc:492] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: No bootstrap required, opened a new log
I20260810 06:43:43.501314 14858 ts_tablet_manager.cc:1397] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:43:43.501497 14864 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.501868 14864 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:43.502146 14858 raft_consensus.cc:348] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.502364 14854 ts_tablet_manager.cc:1428] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 06:43:43.502431 14858 raft_consensus.cc:374] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:43.502516 14858 raft_consensus.cc:729] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c18629ed3134db7b12e2ef43873ecd5, State: Initialized, Role: FOLLOWER
I20260810 06:43:43.502523 14856 tablet_bootstrap.cc:492] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: No bootstrap required, opened a new log
I20260810 06:43:43.502651 14856 ts_tablet_manager.cc:1397] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 06:43:43.502727 14858 consensus_queue.cc:260] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.503111 14866 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.503283 14858 ts_tablet_manager.cc:1428] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:43:43.503422 14866 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:43.503373 14856 raft_consensus.cc:348] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.503631 14856 raft_consensus.cc:374] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:43.503736 14856 raft_consensus.cc:729] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220f89114a334ea2ab42a24dece98c54, State: Initialized, Role: FOLLOWER
I20260810 06:43:43.503962 14856 consensus_queue.cc:260] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.504586 14856 ts_tablet_manager.cc:1428] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:43:43.504434 14869 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.504806 14869 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:43.595607 14869 raft_consensus.cc:482] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:43:43.595849 14869 raft_consensus.cc:504] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.597739 14869 leader_election.cc:288] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535), 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159)
I20260810 06:43:43.604830 14853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.598040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:51157 (local address 127.10.242.1:43535)
0810 06:43:43.598305 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.598311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.598904 (+ 593us) server_negotiation.cc:407] Connection header received
0810 06:43:43.599258 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.599266 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.599350 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.599414 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:43.600171 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.601267 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.603175 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.603928 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.604081 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.604156 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.604446 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.604582 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:43:43.604866 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.598043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:43535 (local address 127.10.242.2:51157)
0810 06:43:43.598807 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.598826 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.599022 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.599631 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.599639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.599666 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:43.600004 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.600016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.601405 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.601410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.603013 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.603026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.604120 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.604126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.604288 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.604321 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.604407 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.604537 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":455,"threads_started":1}
I20260810 06:43:43.606063 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.598956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42159 (local address 127.10.242.2:46403)
0810 06:43:43.599806 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.599825 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.599957 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.600307 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.600311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.600335 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:43.600626 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.600635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.601753 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.601758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.603869 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.603884 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.605344 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.605351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.605534 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.605568 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.605647 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.605766 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":517,"threads_started":1}
I20260810 06:43:43.606078 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.599070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:46403 (local address 127.10.242.3:42159)
0810 06:43:43.599236 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.599242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.599930 (+ 688us) server_negotiation.cc:407] Connection header received
0810 06:43:43.600099 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.600105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.600199 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.600275 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:43.600772 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.601601 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.604027 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.605087 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.605242 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.605306 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.605706 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.605834 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:43:43.606226 14288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ea54cd95c44d4e8663068e8e315cbd" candidate_uuid: "220f89114a334ea2ab42a24dece98c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c18629ed3134db7b12e2ef43873ecd5" is_pre_election: true
I20260810 06:43:43.606671 14729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ea54cd95c44d4e8663068e8e315cbd" candidate_uuid: "220f89114a334ea2ab42a24dece98c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" is_pre_election: true
I20260810 06:43:43.606717 14288 raft_consensus.cc:2454] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 220f89114a334ea2ab42a24dece98c54 in term 0.
I20260810 06:43:43.606930 14729 raft_consensus.cc:2454] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 220f89114a334ea2ab42a24dece98c54 in term 0.
I20260810 06:43:43.607436 14398 leader_election.cc:302] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220f89114a334ea2ab42a24dece98c54, 72ae479d89ed4f7db0d5b888874f8ac6; no voters:
I20260810 06:43:43.607837 14869 raft_consensus.cc:2781] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:43:43.607960 14869 raft_consensus.cc:482] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:43:43.608062 14869 raft_consensus.cc:3037] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:43.610211 14869 raft_consensus.cc:504] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.610741 14869 leader_election.cc:288] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [CANDIDATE]: Term 1 election: Requested vote from peers 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535), 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159)
I20260810 06:43:43.611187 14288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ea54cd95c44d4e8663068e8e315cbd" candidate_uuid: "220f89114a334ea2ab42a24dece98c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c18629ed3134db7b12e2ef43873ecd5"
I20260810 06:43:43.611397 14288 raft_consensus.cc:3037] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:43.611497 14729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ea54cd95c44d4e8663068e8e315cbd" candidate_uuid: "220f89114a334ea2ab42a24dece98c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ae479d89ed4f7db0d5b888874f8ac6"
I20260810 06:43:43.611707 14729 raft_consensus.cc:3037] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:43.614495 14288 raft_consensus.cc:2454] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220f89114a334ea2ab42a24dece98c54 in term 1.
I20260810 06:43:43.614511 14729 raft_consensus.cc:2454] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220f89114a334ea2ab42a24dece98c54 in term 1.
I20260810 06:43:43.614881 14398 leader_election.cc:302] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220f89114a334ea2ab42a24dece98c54, 9c18629ed3134db7b12e2ef43873ecd5; no voters:
I20260810 06:43:43.615141 14869 raft_consensus.cc:2781] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:43.615561 14869 raft_consensus.cc:686] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [term 1 LEADER]: Becoming Leader. State: Replica: 220f89114a334ea2ab42a24dece98c54, State: Running, Role: LEADER
I20260810 06:43:43.616044 14869 consensus_queue.cc:237] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.619613 14889 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: TxnStatusTablet state changed. Reason: New leader 220f89114a334ea2ab42a24dece98c54. Latest consensus state: current_term: 1 leader_uuid: "220f89114a334ea2ab42a24dece98c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.619940 14889 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:43:43.620647 14893 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:43:43.620960 14893 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:43:43.620999 14032 catalog_manager.cc:5495] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 220f89114a334ea2ab42a24dece98c54 (127.10.242.2). New cstate: current_term: 1 leader_uuid: "220f89114a334ea2ab42a24dece98c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } health_report { overall_health: UNKNOWN } } }
I20260810 06:43:43.661309 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.655119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:46529 (local address 127.0.0.1:49630)
0810 06:43:43.655490 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.655506 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.655632 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.656113 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.656116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.656125 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:43:43.656327 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.656337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.657794 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.657799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.658830 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.658843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.659769 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.659773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.659915 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.659967 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:43:43.660909 (+ 942us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:43:43.660919 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.660965 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.661071 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260810 06:43:43.661415 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.655237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.10.242.2:46529)
0810 06:43:43.655427 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.655434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.655568 (+ 134us) server_negotiation.cc:407] Connection header received
0810 06:43:43.655837 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.655844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.655952 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.656115 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:43.656455 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.657673 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.658967 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.659637 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.659766 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.660148 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:43.660817 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:43.660959 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.661012 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.661096 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:43:43.669376 14288 raft_consensus.cc:1261] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Refusing update from remote peer 220f89114a334ea2ab42a24dece98c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:43.669540 14729 raft_consensus.cc:1261] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Refusing update from remote peer 220f89114a334ea2ab42a24dece98c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:43.670480 14889 consensus_queue.cc:1035] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:43.670804 14869 consensus_queue.cc:1035] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:43.680429 14866 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: TxnStatusTablet state changed. Reason: New leader 220f89114a334ea2ab42a24dece98c54. Latest consensus state: current_term: 1 leader_uuid: "220f89114a334ea2ab42a24dece98c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.680761 14866 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:43.682626 14896 mvcc.cc:205] Tried to move back new op lower bound from 7316865940140695552 to 7316865939935354880. Current Snapshot: MvccSnapshot[applied={T|T < 7316865940140695552}]
I20260810 06:43:43.684037 14897 mvcc.cc:205] Tried to move back new op lower bound from 7316865940140695552 to 7316865939935354880. Current Snapshot: MvccSnapshot[applied={T|T < 7316865940140695552}]
I20260810 06:43:43.685137 14864 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: TxnStatusTablet state changed. Reason: New leader 220f89114a334ea2ab42a24dece98c54. Latest consensus state: current_term: 1 leader_uuid: "220f89114a334ea2ab42a24dece98c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.685405 14864 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:43.686801 14866 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "220f89114a334ea2ab42a24dece98c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.687042 14866 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 9c18629ed3134db7b12e2ef43873ecd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:43.689576 14869 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "220f89114a334ea2ab42a24dece98c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.689862 14869 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:43:43.690069 14869 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "220f89114a334ea2ab42a24dece98c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.690299 14869 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 220f89114a334ea2ab42a24dece98c54: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:43:43.692876 14864 tablet_replica.cc:439] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "220f89114a334ea2ab42a24dece98c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } } }
I20260810 06:43:43.693112 14864 tablet_replica.cc:442] T 10ea54cd95c44d4e8663068e8e315cbd P 72ae479d89ed4f7db0d5b888874f8ac6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:43:43.722549 14853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.715425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.10.242.1:43535)
0810 06:43:43.715591 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.715597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.715931 (+ 334us) server_negotiation.cc:407] Connection header received
0810 06:43:43.716154 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.716160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.716281 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.716423 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:43.716811 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.717941 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.719973 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.720599 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.720714 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.721273 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:43.721906 (+ 633us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:43.722037 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.722141 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.722257 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:43:43.739012 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.732904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:40183 (local address 127.10.242.2:46529)
0810 06:43:43.733123 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.733129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.733686 (+ 557us) server_negotiation.cc:407] Connection header received
0810 06:43:43.733885 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.733889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.733954 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.734025 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:43.734599 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.735352 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.737292 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.737729 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.737845 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.738393 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:43.738521 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:43.738590 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.738711 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.738775 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:43:43.739440 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.732841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:46529 (local address 127.10.242.1:40183)
0810 06:43:43.733579 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.733599 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.733744 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.734057 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.734062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.734075 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:43:43.734464 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.734476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.735471 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.735476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.737166 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.737181 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.737892 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.737899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.738107 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.738209 (+ 102us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:43:43.738609 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:43:43.738620 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.738699 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.738839 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":324,"threads_started":1}
I20260810 06:43:43.748648 14729 raft_consensus.cc:1261] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Refusing update from remote peer 9c18629ed3134db7b12e2ef43873ecd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:43.749190 14501 raft_consensus.cc:1261] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Refusing update from remote peer 9c18629ed3134db7b12e2ef43873ecd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:43.749243 14874 consensus_queue.cc:1035] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:43.750039 14874 consensus_queue.cc:1035] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:43.786711 14853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.779777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:35871 (local address 127.10.242.1:43535)
0810 06:43:43.779970 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:43.779977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:43.780670 (+ 693us) server_negotiation.cc:407] Connection header received
0810 06:43:43.780742 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:43.780747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:43.780831 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:43.780921 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:43.781568 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.782568 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.784770 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.785325 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.785431 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.785993 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:43.786094 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:43.786210 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:43.786335 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 06:43:43.786409 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:43:43.786950 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:43.779753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:43535 (local address 127.10.242.2:35871)
0810 06:43:43.780449 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:43.780468 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:43.780604 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:43.780949 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:43.780954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:43.780966 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:43:43.781376 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.781389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.782688 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.782694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.784614 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:43.784634 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:43.785499 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:43.785507 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:43.785663 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:43.785724 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:43:43.786211 (+ 487us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:43:43.786232 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 06:43:43.786313 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:43:43.786438 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":356,"threads_started":1}
I20260810 06:43:43.822636 14029 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:43.823006 14029 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:43.827143 14029 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a: Sending DeleteTablet for 3 replicas of tablet 73b822f4d990406483a0d90649c820a3
I20260810 06:43:43.828228 14696 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b822f4d990406483a0d90649c820a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:43 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:56792
I20260810 06:43:43.828545 14473 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b822f4d990406483a0d90649c820a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:43 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:49614
I20260810 06:43:43.829751 14258 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b822f4d990406483a0d90649c820a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:43 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:52416
I20260810 06:43:43.834440 14029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:43:43.835108 14029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:43:43.840955 14942 tablet_replica.cc:330] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: stopping tablet replica
I20260810 06:43:43.841552 14942 raft_consensus.cc:2229] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:43.841990 14942 raft_consensus.cc:2258] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:43.844033 14942 ts_tablet_manager.cc:1905] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:43.846879 14472 tablet_service.cc:1467] Processing CreateTablet for tablet 6c2f8b147ebe4253a4493abb45d63b33 (DEFAULT_TABLE table=test-table [id=3bce7885e5354b36bbe6ea45968d012d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:43.847398 14472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c2f8b147ebe4253a4493abb45d63b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:43.848136 14696 tablet_service.cc:1467] Processing CreateTablet for tablet 6c2f8b147ebe4253a4493abb45d63b33 (DEFAULT_TABLE table=test-table [id=3bce7885e5354b36bbe6ea45968d012d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:43.850664 14257 tablet_service.cc:1467] Processing CreateTablet for tablet 6c2f8b147ebe4253a4493abb45d63b33 (DEFAULT_TABLE table=test-table [id=3bce7885e5354b36bbe6ea45968d012d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:43.851250 14257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c2f8b147ebe4253a4493abb45d63b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:43.861044 14858 tablet_bootstrap.cc:492] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: Bootstrap starting.
I20260810 06:43:43.862715 14856 tablet_bootstrap.cc:492] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: Bootstrap starting.
I20260810 06:43:43.862797 14858 tablet_bootstrap.cc:654] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:43.864311 14856 tablet_bootstrap.cc:654] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:43.867385 14946 tablet_replica.cc:330] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: stopping tablet replica
I20260810 06:43:43.867830 14946 raft_consensus.cc:2229] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:43:43.868515 14946 raft_consensus.cc:2258] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:43.869328 14696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c2f8b147ebe4253a4493abb45d63b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:43.869340 14944 tablet_replica.cc:330] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: stopping tablet replica
I20260810 06:43:43.869731 14944 raft_consensus.cc:2229] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:43.870035 14944 raft_consensus.cc:2258] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:43.872985 14946 ts_tablet_manager.cc:1905] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:43.875995 14854 tablet_bootstrap.cc:492] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: Bootstrap starting.
I20260810 06:43:43.877552 14854 tablet_bootstrap.cc:654] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:43.880342 14944 ts_tablet_manager.cc:1905] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:43.882326 14858 tablet_bootstrap.cc:492] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: No bootstrap required, opened a new log
I20260810 06:43:43.882506 14858 ts_tablet_manager.cc:1397] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.005s
I20260810 06:43:43.883286 14858 raft_consensus.cc:348] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.883541 14858 raft_consensus.cc:374] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:43.883616 14858 raft_consensus.cc:729] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c18629ed3134db7b12e2ef43873ecd5, State: Initialized, Role: FOLLOWER
I20260810 06:43:43.883929 14858 consensus_queue.cc:260] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.886961 14858 ts_tablet_manager.cc:1428] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 06:43:43.889909 14944 ts_tablet_manager.cc:1918] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:43:43.890084 14944 log.cc:1196] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/73b822f4d990406483a0d90649c820a3
I20260810 06:43:43.892499 14946 ts_tablet_manager.cc:1918] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:43:43.892680 14946 log.cc:1196] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/73b822f4d990406483a0d90649c820a3
I20260810 06:43:43.893227 14946 ts_tablet_manager.cc:1939] T 73b822f4d990406483a0d90649c820a3 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting consensus metadata
I20260810 06:43:43.894943 14004 catalog_manager.cc:4841] TS 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535): tablet 73b822f4d990406483a0d90649c820a3 (table test-table [id=a041dfebf6a445ffbbe86c1c64308993]) successfully deleted
I20260810 06:43:43.895687 14944 ts_tablet_manager.cc:1939] T 73b822f4d990406483a0d90649c820a3 P 220f89114a334ea2ab42a24dece98c54: Deleting consensus metadata
I20260810 06:43:43.896389 14854 tablet_bootstrap.cc:492] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: No bootstrap required, opened a new log
I20260810 06:43:43.896603 14854 ts_tablet_manager.cc:1397] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.004s
I20260810 06:43:43.897363 14854 raft_consensus.cc:348] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.897650 14854 raft_consensus.cc:374] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:43.897770 14854 raft_consensus.cc:729] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ae479d89ed4f7db0d5b888874f8ac6, State: Initialized, Role: FOLLOWER
I20260810 06:43:43.898077 14854 consensus_queue.cc:260] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.899215 14004 catalog_manager.cc:4841] TS 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529): tablet 73b822f4d990406483a0d90649c820a3 (table test-table [id=a041dfebf6a445ffbbe86c1c64308993]) successfully deleted
I20260810 06:43:43.899570 14942 ts_tablet_manager.cc:1918] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:43:43.899739 14942 log.cc:1196] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/73b822f4d990406483a0d90649c820a3
I20260810 06:43:43.900048 14856 tablet_bootstrap.cc:492] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: No bootstrap required, opened a new log
I20260810 06:43:43.900213 14856 ts_tablet_manager.cc:1397] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: Time spent bootstrapping tablet: real 0.038s user 0.007s sys 0.005s
I20260810 06:43:43.900220 14942 ts_tablet_manager.cc:1939] T 73b822f4d990406483a0d90649c820a3 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting consensus metadata
I20260810 06:43:43.900902 14856 raft_consensus.cc:348] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.901132 14856 raft_consensus.cc:374] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:43.901217 14856 raft_consensus.cc:729] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220f89114a334ea2ab42a24dece98c54, State: Initialized, Role: FOLLOWER
I20260810 06:43:43.901427 14856 consensus_queue.cc:260] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:43.901793 14004 catalog_manager.cc:4841] TS 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159): tablet 73b822f4d990406483a0d90649c820a3 (table test-table [id=a041dfebf6a445ffbbe86c1c64308993]) successfully deleted
I20260810 06:43:43.901998 14856 ts_tablet_manager.cc:1428] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:43:43.902176 14854 ts_tablet_manager.cc:1428] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:43:44.114053 14911 raft_consensus.cc:482] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:43:44.114307 14911 raft_consensus.cc:504] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:44.114938 14911 leader_election.cc:288] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535), 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159)
I20260810 06:43:44.115687 14729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2f8b147ebe4253a4493abb45d63b33" candidate_uuid: "220f89114a334ea2ab42a24dece98c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" is_pre_election: true
I20260810 06:43:44.115670 14288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2f8b147ebe4253a4493abb45d63b33" candidate_uuid: "220f89114a334ea2ab42a24dece98c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c18629ed3134db7b12e2ef43873ecd5" is_pre_election: true
I20260810 06:43:44.115916 14729 raft_consensus.cc:2454] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 220f89114a334ea2ab42a24dece98c54 in term 0.
I20260810 06:43:44.115970 14288 raft_consensus.cc:2454] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 220f89114a334ea2ab42a24dece98c54 in term 0.
I20260810 06:43:44.116410 14398 leader_election.cc:302] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220f89114a334ea2ab42a24dece98c54, 72ae479d89ed4f7db0d5b888874f8ac6; no voters:
I20260810 06:43:44.116698 14911 raft_consensus.cc:2781] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:43:44.116813 14911 raft_consensus.cc:482] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:43:44.116904 14911 raft_consensus.cc:3037] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:44.119279 14911 raft_consensus.cc:504] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:44.119791 14911 leader_election.cc:288] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [CANDIDATE]: Term 1 election: Requested vote from peers 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535), 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159)
I20260810 06:43:44.120383 14729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2f8b147ebe4253a4493abb45d63b33" candidate_uuid: "220f89114a334ea2ab42a24dece98c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ae479d89ed4f7db0d5b888874f8ac6"
I20260810 06:43:44.120353 14288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c2f8b147ebe4253a4493abb45d63b33" candidate_uuid: "220f89114a334ea2ab42a24dece98c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c18629ed3134db7b12e2ef43873ecd5"
I20260810 06:43:44.120568 14729 raft_consensus.cc:3037] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:44.120580 14288 raft_consensus.cc:3037] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:44.122617 14729 raft_consensus.cc:2454] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220f89114a334ea2ab42a24dece98c54 in term 1.
I20260810 06:43:44.123009 14398 leader_election.cc:302] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220f89114a334ea2ab42a24dece98c54, 72ae479d89ed4f7db0d5b888874f8ac6; no voters:
I20260810 06:43:44.123286 14911 raft_consensus.cc:2781] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:44.123453 14911 raft_consensus.cc:686] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 1 LEADER]: Becoming Leader. State: Replica: 220f89114a334ea2ab42a24dece98c54, State: Running, Role: LEADER
I20260810 06:43:44.123821 14288 raft_consensus.cc:2454] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220f89114a334ea2ab42a24dece98c54 in term 1.
I20260810 06:43:44.123764 14911 consensus_queue.cc:237] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:44.126824 14029 catalog_manager.cc:5495] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 220f89114a334ea2ab42a24dece98c54 (127.10.242.2). New cstate: current_term: 1 leader_uuid: "220f89114a334ea2ab42a24dece98c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } health_report { overall_health: UNKNOWN } } }
I20260810 06:43:44.159713 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:44.154362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.10.242.2:46529)
0810 06:43:44.154563 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:44.154569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:44.154590 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:43:44.154693 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:44.154699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:44.154787 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:44.154885 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:44.155316 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:44.156453 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:44.157729 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:44.158446 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:44.158582 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:44.158972 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:44.159126 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:44.159223 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:44.159335 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 06:43:44.159416 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:43:44.173182 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:44.168055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:46529 (local address 127.10.242.2:40097)
0810 06:43:44.168296 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:44.168311 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:44.168393 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:44.168672 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:44.168675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:44.168684 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:43:44.168852 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:44.168858 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:44.170212 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:44.170216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:44.171234 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:44.171246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:44.172175 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:44.172179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:44.172301 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:44.172337 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:43:44.172743 (+ 406us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:43:44.172752 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:43:44.172812 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 06:43:44.172911 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260810 06:43:44.173259 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:44.168196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:40097 (local address 127.10.242.2:46529)
0810 06:43:44.168373 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:44.168379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:44.168399 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:43:44.168483 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:44.168488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:44.168569 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:44.168667 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:44.168966 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:44.170027 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:44.171369 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:44.171979 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:44.172158 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:44.172505 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:44.172644 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:44.172743 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:44.172854 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 06:43:44.172931 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":53}
I20260810 06:43:44.181149 14729 raft_consensus.cc:1261] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Refusing update from remote peer 220f89114a334ea2ab42a24dece98c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:44.181308 14286 raft_consensus.cc:1261] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Refusing update from remote peer 220f89114a334ea2ab42a24dece98c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:44.181815 14889 consensus_queue.cc:1035] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:44.182099 14911 consensus_queue.cc:1035] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:43:44.206005 14896 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:43:45.835498 14032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:45.835739 14032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:45.839753 14032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a: Sending DeleteTablet for 3 replicas of tablet 6c2f8b147ebe4253a4493abb45d63b33
I20260810 06:43:45.840492 14258 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c2f8b147ebe4253a4493abb45d63b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:45 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:52416
I20260810 06:43:45.840575 14473 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c2f8b147ebe4253a4493abb45d63b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:45 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:49614
I20260810 06:43:45.840672 14696 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c2f8b147ebe4253a4493abb45d63b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:45 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:56792
I20260810 06:43:45.852856 14995 tablet_replica.cc:330] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: stopping tablet replica
I20260810 06:43:45.853223 14995 raft_consensus.cc:2229] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:45.853543 14995 raft_consensus.cc:2258] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:45.854754 14995 ts_tablet_manager.cc:1905] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:45.855244 14996 tablet_replica.cc:330] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: stopping tablet replica
I20260810 06:43:45.855515 14996 raft_consensus.cc:2229] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:43:45.856194 14996 raft_consensus.cc:2258] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:45.857159 14997 tablet_replica.cc:330] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: stopping tablet replica
I20260810 06:43:45.857276 14996 ts_tablet_manager.cc:1905] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:45.857378 14997 raft_consensus.cc:2229] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:45.857622 14997 raft_consensus.cc:2258] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:45.858671 14997 ts_tablet_manager.cc:1905] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:45.861341 14995 ts_tablet_manager.cc:1918] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:43:45.861511 14995 log.cc:1196] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/6c2f8b147ebe4253a4493abb45d63b33
I20260810 06:43:45.861960 14995 ts_tablet_manager.cc:1939] T 6c2f8b147ebe4253a4493abb45d63b33 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting consensus metadata
I20260810 06:43:45.863135 14004 catalog_manager.cc:4841] TS 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535): tablet 6c2f8b147ebe4253a4493abb45d63b33 (table test-table [id=3bce7885e5354b36bbe6ea45968d012d]) successfully deleted
I20260810 06:43:45.863334 14996 ts_tablet_manager.cc:1918] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:43:45.863528 14996 log.cc:1196] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/6c2f8b147ebe4253a4493abb45d63b33
I20260810 06:43:45.863919 14996 ts_tablet_manager.cc:1939] T 6c2f8b147ebe4253a4493abb45d63b33 P 220f89114a334ea2ab42a24dece98c54: Deleting consensus metadata
I20260810 06:43:45.864594 14997 ts_tablet_manager.cc:1918] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:43:45.864717 14997 log.cc:1196] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/6c2f8b147ebe4253a4493abb45d63b33
I20260810 06:43:45.864980 14004 catalog_manager.cc:4841] TS 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529): tablet 6c2f8b147ebe4253a4493abb45d63b33 (table test-table [id=3bce7885e5354b36bbe6ea45968d012d]) successfully deleted
I20260810 06:43:45.865065 14997 ts_tablet_manager.cc:1939] T 6c2f8b147ebe4253a4493abb45d63b33 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting consensus metadata
I20260810 06:43:45.865918 14004 catalog_manager.cc:4841] TS 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159): tablet 6c2f8b147ebe4253a4493abb45d63b33 (table test-table [id=3bce7885e5354b36bbe6ea45968d012d]) successfully deleted
Aug 10 06:43:45 dist-test-slave-zj8q krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344225, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:43:45 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344225, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:45.900954 15009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:45.887893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.10.242.62:34407)
0810 06:43:45.888411 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:45.888416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:45.888437 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:43:45.888584 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:45.888588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:45.888651 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:45.888740 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:45.889978 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.890821 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.892039 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.892422 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.892511 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:45.896011 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:45.896016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:45.896019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:45.896052 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:45.898719 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:45.899255 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:45.899258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:45.899261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:45.899374 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:45.899623 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:45.899628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:45.899630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:45.899872 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:45.900103 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:45.900412 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 06:43:45.900624 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":260,"threads_started":1}
I20260810 06:43:45.904065 14032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:43:45.904640 14032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:43:45.910640 14258 tablet_service.cc:1467] Processing CreateTablet for tablet f7ce0e592e804fcb88168150bacb5f96 (DEFAULT_TABLE table=test-table [id=ee56ec287116445fa27022ba93b1f491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:45.910797 14473 tablet_service.cc:1467] Processing CreateTablet for tablet f7ce0e592e804fcb88168150bacb5f96 (DEFAULT_TABLE table=test-table [id=ee56ec287116445fa27022ba93b1f491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:45.910894 14696 tablet_service.cc:1467] Processing CreateTablet for tablet f7ce0e592e804fcb88168150bacb5f96 (DEFAULT_TABLE table=test-table [id=ee56ec287116445fa27022ba93b1f491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:45.911100 14258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ce0e592e804fcb88168150bacb5f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:45.911172 14473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ce0e592e804fcb88168150bacb5f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:45.911324 14696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ce0e592e804fcb88168150bacb5f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:45.917868 15011 tablet_bootstrap.cc:492] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: Bootstrap starting.
I20260810 06:43:45.918715 15014 tablet_bootstrap.cc:492] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: Bootstrap starting.
I20260810 06:43:45.919325 15013 tablet_bootstrap.cc:492] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: Bootstrap starting.
I20260810 06:43:45.919852 15011 tablet_bootstrap.cc:654] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:45.920496 15014 tablet_bootstrap.cc:654] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:45.921028 15013 tablet_bootstrap.cc:654] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:45.922138 15011 tablet_bootstrap.cc:492] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: No bootstrap required, opened a new log
I20260810 06:43:45.922305 15011 ts_tablet_manager.cc:1397] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260810 06:43:45.922956 15013 tablet_bootstrap.cc:492] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: No bootstrap required, opened a new log
I20260810 06:43:45.923110 15014 tablet_bootstrap.cc:492] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: No bootstrap required, opened a new log
I20260810 06:43:45.923148 15013 ts_tablet_manager.cc:1397] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:43:45.923054 15011 raft_consensus.cc:348] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:45.923300 15014 ts_tablet_manager.cc:1397] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 06:43:45.923300 15011 raft_consensus.cc:374] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:45.923393 15011 raft_consensus.cc:729] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ae479d89ed4f7db0d5b888874f8ac6, State: Initialized, Role: FOLLOWER
I20260810 06:43:45.923627 15011 consensus_queue.cc:260] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:45.924034 15013 raft_consensus.cc:348] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:45.924057 15014 raft_consensus.cc:348] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:45.924274 15013 raft_consensus.cc:374] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:45.924302 15014 raft_consensus.cc:374] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:45.924386 15014 raft_consensus.cc:729] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220f89114a334ea2ab42a24dece98c54, State: Initialized, Role: FOLLOWER
I20260810 06:43:45.924377 15013 raft_consensus.cc:729] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c18629ed3134db7b12e2ef43873ecd5, State: Initialized, Role: FOLLOWER
I20260810 06:43:45.924615 15013 consensus_queue.cc:260] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:45.924639 15014 consensus_queue.cc:260] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:45.925877 15014 ts_tablet_manager.cc:1428] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:43:45.926517 15011 ts_tablet_manager.cc:1428] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260810 06:43:45.927062 15013 ts_tablet_manager.cc:1428] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260810 06:43:45.936332 15020 raft_consensus.cc:482] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:43:45.936535 15020 raft_consensus.cc:504] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:45.937949 15020 leader_election.cc:288] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535), 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529)
I20260810 06:43:45.945087 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:45.938346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:52289 (local address 127.10.242.1:43535)
0810 06:43:45.938874 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:45.938880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:45.939271 (+ 391us) server_negotiation.cc:407] Connection header received
0810 06:43:45.939389 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:45.939394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:45.939467 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:45.940194 (+ 727us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:45.940342 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.941097 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.943017 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.944173 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.944303 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:45.944368 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:45.944722 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 06:43:45.944812 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":258,"threads_started":1}
I20260810 06:43:45.945181 15028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:45.938276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:43535 (local address 127.10.242.3:52289)
0810 06:43:45.939111 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:45.939130 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:45.939265 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:45.939681 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:45.939687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:45.939725 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:45.940047 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:45.940058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.941222 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.941227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:45.942888 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:45.942900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.944387 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.944393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:45.944577 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:45.944613 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:43:45.944707 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 06:43:45.944856 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":582,"threads_started":1}
I20260810 06:43:45.945969 14286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ce0e592e804fcb88168150bacb5f96" candidate_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c18629ed3134db7b12e2ef43873ecd5" is_pre_election: true
I20260810 06:43:45.946264 14286 raft_consensus.cc:2454] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ae479d89ed4f7db0d5b888874f8ac6 in term 0.
I20260810 06:43:45.946359 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:45.939256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:46529 (local address 127.10.242.3:56653)
0810 06:43:45.939928 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:45.939944 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:45.940069 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:45.940392 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:45.940397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:45.940420 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:43:45.940717 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:45.940726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.942052 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.942056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:45.943858 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:45.943874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.945557 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.945565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:45.945744 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:45.945775 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:43:45.945859 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:43:45.945972 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":1,"thread_start_us":311,"threads_started":1}
I20260810 06:43:45.946765 14619 leader_election.cc:302] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ae479d89ed4f7db0d5b888874f8ac6, 9c18629ed3134db7b12e2ef43873ecd5; no voters:
I20260810 06:43:45.946898 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:45.939398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:56653 (local address 127.10.242.2:46529)
0810 06:43:45.940148 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:45.940154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:45.940174 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:43:45.940226 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:45.940230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:45.940279 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:45.940356 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:43:45.940872 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.941924 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.944006 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.945139 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.945317 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:45.945399 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:45.946594 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0810 06:43:45.946694 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":458,"threads_started":1}
I20260810 06:43:45.947108 15020 raft_consensus.cc:2781] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:43:45.947230 15020 raft_consensus.cc:482] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:43:45.947333 15020 raft_consensus.cc:3037] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:45.947535 14501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ce0e592e804fcb88168150bacb5f96" candidate_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220f89114a334ea2ab42a24dece98c54" is_pre_election: true
I20260810 06:43:45.947762 14501 raft_consensus.cc:2454] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ae479d89ed4f7db0d5b888874f8ac6 in term 0.
I20260810 06:43:45.949681 15020 raft_consensus.cc:504] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:45.950220 15020 leader_election.cc:288] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [CANDIDATE]: Term 1 election: Requested vote from peers 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535), 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529)
I20260810 06:43:45.950603 14286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ce0e592e804fcb88168150bacb5f96" candidate_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c18629ed3134db7b12e2ef43873ecd5"
I20260810 06:43:45.950794 14286 raft_consensus.cc:3037] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:45.950731 14501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ce0e592e804fcb88168150bacb5f96" candidate_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220f89114a334ea2ab42a24dece98c54"
I20260810 06:43:45.950923 14501 raft_consensus.cc:3037] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:45.952702 14286 raft_consensus.cc:2454] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ae479d89ed4f7db0d5b888874f8ac6 in term 1.
I20260810 06:43:45.953029 14619 leader_election.cc:302] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ae479d89ed4f7db0d5b888874f8ac6, 9c18629ed3134db7b12e2ef43873ecd5; no voters:
I20260810 06:43:45.953091 14501 raft_consensus.cc:2454] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ae479d89ed4f7db0d5b888874f8ac6 in term 1.
I20260810 06:43:45.953291 15020 raft_consensus.cc:2781] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:45.953919 15020 raft_consensus.cc:686] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 LEADER]: Becoming Leader. State: Replica: 72ae479d89ed4f7db0d5b888874f8ac6, State: Running, Role: LEADER
I20260810 06:43:45.954327 15020 consensus_queue.cc:237] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:45.957297 14024 catalog_manager.cc:5495] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3). New cstate: current_term: 1 leader_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } health_report { overall_health: HEALTHY } } }
I20260810 06:43:45.982031 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:45.976212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56800 (local address 127.10.242.3:42159)
0810 06:43:45.976834 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:45.976838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:45.976854 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:43:45.976930 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:45.976934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:45.977006 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:45.977163 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:45.977581 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.978481 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.979789 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.980206 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.980312 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:45.980771 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:45.981382 (+ 611us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:45.981515 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:45.981644 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 06:43:45.981718 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":378,"threads_started":1}
I20260810 06:43:45.996244 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:45.991153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:51251 (local address 127.10.242.2:46529)
0810 06:43:45.991339 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:45.991346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:45.991696 (+ 350us) server_negotiation.cc:407] Connection header received
0810 06:43:45.991863 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:45.991867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:45.991929 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:45.991999 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:45.992473 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.993173 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.994578 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.995005 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.995128 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:45.995551 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:45.995679 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:45.995755 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:45.995885 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 06:43:45.995964 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:43:45.996348 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:45.990964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:46529 (local address 127.10.242.3:51251)
0810 06:43:45.991599 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:45.991614 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:45.991741 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:45.992045 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:45.992050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:45.992062 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:43:45.992335 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:45.992343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.993304 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.993308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:45.994389 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:45.994402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:45.995212 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:45.995217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:45.995336 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:45.995385 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:43:45.995778 (+ 393us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:43:45.995789 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:43:45.995853 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 06:43:45.995978 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":324,"threads_started":1}
I20260810 06:43:46.003870 14501 raft_consensus.cc:1261] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Refusing update from remote peer 72ae479d89ed4f7db0d5b888874f8ac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:46.003935 14286 raft_consensus.cc:1261] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Refusing update from remote peer 72ae479d89ed4f7db0d5b888874f8ac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:46.004529 15032 consensus_queue.cc:1035] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:46.004892 15020 consensus_queue.cc:1035] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:46.010252 14967 mvcc.cc:205] Tried to move back new op lower bound from 7316865949706727424 to 7316865949512417280. Current Snapshot: MvccSnapshot[applied={T|T < 7316865949706727424}]
I20260810 06:43:46.034879 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:46.029503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:43335 (local address 127.10.242.3:42159)
0810 06:43:46.029660 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:46.029667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:46.030099 (+ 432us) server_negotiation.cc:407] Connection header received
0810 06:43:46.030307 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:46.030311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:46.030370 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:46.030437 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:43:46.030802 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:46.031591 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:46.033147 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:46.033653 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:46.033760 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:46.034147 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:43:46.034281 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:43:46.034356 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:46.034487 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 06:43:46.034562 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:43:46.034932 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:46.029418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42159 (local address 127.10.242.2:43335)
0810 06:43:46.030000 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:46.030018 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:46.030185 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:46.030449 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:46.030452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:46.030460 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:43:46.030679 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:46.030687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:46.031722 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:46.031727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:46.033012 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:46.033025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:46.033821 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:46.033828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:46.033966 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:46.034001 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:43:46.034372 (+ 371us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:43:46.034384 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 06:43:46.034457 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:43:46.034586 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":303,"threads_started":1}
I20260810 06:43:46.061231 14024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:46.061461 14024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:46.064321 14024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a: Sending DeleteTablet for 3 replicas of tablet f7ce0e592e804fcb88168150bacb5f96
I20260810 06:43:46.065165 14258 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ce0e592e804fcb88168150bacb5f96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:46 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:52416
I20260810 06:43:46.065156 14473 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ce0e592e804fcb88168150bacb5f96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:46 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:49614
I20260810 06:43:46.065430 14995 tablet_replica.cc:330] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: stopping tablet replica
I20260810 06:43:46.065598 14995 raft_consensus.cc:2229] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:46.065755 14995 raft_consensus.cc:2258] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:46.065837 14996 tablet_replica.cc:330] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: stopping tablet replica
I20260810 06:43:46.066059 14996 raft_consensus.cc:2229] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:46.066295 14996 raft_consensus.cc:2258] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:46.066349 14024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 06:43:46.066731 14696 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ce0e592e804fcb88168150bacb5f96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:46 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:56792
W20260810 06:43:46.066983 14024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:43:46.067536 14996 ts_tablet_manager.cc:1905] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:46.067946 14995 ts_tablet_manager.cc:1905] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:46.068131 14997 tablet_replica.cc:330] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: stopping tablet replica
I20260810 06:43:46.068329 14997 raft_consensus.cc:2229] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:43:46.068827 14997 raft_consensus.cc:2258] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:46.073513 14995 ts_tablet_manager.cc:1918] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:43:46.073655 14995 log.cc:1196] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/f7ce0e592e804fcb88168150bacb5f96
I20260810 06:43:46.074172 14995 ts_tablet_manager.cc:1939] T f7ce0e592e804fcb88168150bacb5f96 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting consensus metadata
I20260810 06:43:46.074142 14996 ts_tablet_manager.cc:1918] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:43:46.075347 14004 catalog_manager.cc:4841] TS 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535): tablet f7ce0e592e804fcb88168150bacb5f96 (table test-table [id=ee56ec287116445fa27022ba93b1f491]) successfully deleted
I20260810 06:43:46.075723 14997 ts_tablet_manager.cc:1905] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:46.074311 14996 log.cc:1196] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/f7ce0e592e804fcb88168150bacb5f96
I20260810 06:43:46.076983 14996 ts_tablet_manager.cc:1939] T f7ce0e592e804fcb88168150bacb5f96 P 220f89114a334ea2ab42a24dece98c54: Deleting consensus metadata
I20260810 06:43:46.078338 14258 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac007e092d34715aefc3d24daa6a9b1 (DEFAULT_TABLE table=test-table [id=2b5b67dfee6a4ddbb027d9ec88bc6d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:46.078621 14473 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac007e092d34715aefc3d24daa6a9b1 (DEFAULT_TABLE table=test-table [id=2b5b67dfee6a4ddbb027d9ec88bc6d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:46.078742 14258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac007e092d34715aefc3d24daa6a9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:46.079006 14473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac007e092d34715aefc3d24daa6a9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:46.082553 14004 catalog_manager.cc:4841] TS 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529): tablet f7ce0e592e804fcb88168150bacb5f96 (table test-table [id=ee56ec287116445fa27022ba93b1f491]) successfully deleted
I20260810 06:43:46.083338 14997 ts_tablet_manager.cc:1918] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:43:46.083561 14997 log.cc:1196] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/f7ce0e592e804fcb88168150bacb5f96
I20260810 06:43:46.083837 14696 tablet_service.cc:1467] Processing CreateTablet for tablet 4ac007e092d34715aefc3d24daa6a9b1 (DEFAULT_TABLE table=test-table [id=2b5b67dfee6a4ddbb027d9ec88bc6d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:43:46.084173 14997 ts_tablet_manager.cc:1939] T f7ce0e592e804fcb88168150bacb5f96 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting consensus metadata
I20260810 06:43:46.084250 14696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ac007e092d34715aefc3d24daa6a9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:46.085325 15013 tablet_bootstrap.cc:492] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: Bootstrap starting.
I20260810 06:43:46.087010 15013 tablet_bootstrap.cc:654] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:46.088195 15014 tablet_bootstrap.cc:492] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: Bootstrap starting.
I20260810 06:43:46.089691 15014 tablet_bootstrap.cc:654] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:46.090862 14004 catalog_manager.cc:4841] TS 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159): tablet f7ce0e592e804fcb88168150bacb5f96 (table test-table [id=ee56ec287116445fa27022ba93b1f491]) successfully deleted
I20260810 06:43:46.091514 15011 tablet_bootstrap.cc:492] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: Bootstrap starting.
I20260810 06:43:46.092682 15013 tablet_bootstrap.cc:492] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: No bootstrap required, opened a new log
I20260810 06:43:46.092849 15013 ts_tablet_manager.cc:1397] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260810 06:43:46.093050 15011 tablet_bootstrap.cc:654] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:46.093464 15014 tablet_bootstrap.cc:492] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: No bootstrap required, opened a new log
I20260810 06:43:46.093590 15014 ts_tablet_manager.cc:1397] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 06:43:46.093583 15013 raft_consensus.cc:348] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:46.093835 15013 raft_consensus.cc:374] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:46.093926 15013 raft_consensus.cc:729] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c18629ed3134db7b12e2ef43873ecd5, State: Initialized, Role: FOLLOWER
I20260810 06:43:46.094239 15014 raft_consensus.cc:348] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:46.094342 15013 consensus_queue.cc:260] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:46.094578 15014 raft_consensus.cc:374] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:46.094715 15014 raft_consensus.cc:729] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220f89114a334ea2ab42a24dece98c54, State: Initialized, Role: FOLLOWER
I20260810 06:43:46.094947 15013 ts_tablet_manager.cc:1428] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 06:43:46.094937 15014 consensus_queue.cc:260] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:46.095220 15011 tablet_bootstrap.cc:492] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: No bootstrap required, opened a new log
I20260810 06:43:46.095371 15011 ts_tablet_manager.cc:1397] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260810 06:43:46.095436 15014 ts_tablet_manager.cc:1428] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:43:46.096066 15011 raft_consensus.cc:348] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:46.096935 15011 raft_consensus.cc:374] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:46.097059 15011 raft_consensus.cc:729] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ae479d89ed4f7db0d5b888874f8ac6, State: Initialized, Role: FOLLOWER
I20260810 06:43:46.097370 15011 consensus_queue.cc:260] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:46.098897 15011 ts_tablet_manager.cc:1428] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260810 06:43:46.200150 15032 raft_consensus.cc:482] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:43:46.200368 15032 raft_consensus.cc:504] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:46.201049 15032 leader_election.cc:288] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535), 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529)
I20260810 06:43:46.201690 14286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac007e092d34715aefc3d24daa6a9b1" candidate_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c18629ed3134db7b12e2ef43873ecd5" is_pre_election: true
I20260810 06:43:46.201742 14501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac007e092d34715aefc3d24daa6a9b1" candidate_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220f89114a334ea2ab42a24dece98c54" is_pre_election: true
I20260810 06:43:46.201970 14501 raft_consensus.cc:2454] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ae479d89ed4f7db0d5b888874f8ac6 in term 0.
I20260810 06:43:46.201996 14286 raft_consensus.cc:2454] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ae479d89ed4f7db0d5b888874f8ac6 in term 0.
I20260810 06:43:46.202440 14619 leader_election.cc:302] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ae479d89ed4f7db0d5b888874f8ac6, 9c18629ed3134db7b12e2ef43873ecd5; no voters:
I20260810 06:43:46.202721 15032 raft_consensus.cc:2781] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:43:46.202834 15032 raft_consensus.cc:482] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:43:46.202952 15032 raft_consensus.cc:3037] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:46.205310 15032 raft_consensus.cc:504] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:46.205922 15032 leader_election.cc:288] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [CANDIDATE]: Term 1 election: Requested vote from peers 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535), 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529)
I20260810 06:43:46.206404 14286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac007e092d34715aefc3d24daa6a9b1" candidate_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c18629ed3134db7b12e2ef43873ecd5"
I20260810 06:43:46.206470 14501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ac007e092d34715aefc3d24daa6a9b1" candidate_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220f89114a334ea2ab42a24dece98c54"
I20260810 06:43:46.206621 14286 raft_consensus.cc:3037] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:46.206696 14501 raft_consensus.cc:3037] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:46.208652 14286 raft_consensus.cc:2454] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ae479d89ed4f7db0d5b888874f8ac6 in term 1.
I20260810 06:43:46.209650 14501 raft_consensus.cc:2454] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ae479d89ed4f7db0d5b888874f8ac6 in term 1.
I20260810 06:43:46.210145 14619 leader_election.cc:302] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ae479d89ed4f7db0d5b888874f8ac6, 9c18629ed3134db7b12e2ef43873ecd5; no voters:
I20260810 06:43:46.210697 15032 raft_consensus.cc:2781] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:46.210862 15032 raft_consensus.cc:686] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 LEADER]: Becoming Leader. State: Replica: 72ae479d89ed4f7db0d5b888874f8ac6, State: Running, Role: LEADER
I20260810 06:43:46.211153 15032 consensus_queue.cc:237] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } }
I20260810 06:43:46.217545 14032 catalog_manager.cc:5495] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3). New cstate: current_term: 1 leader_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72ae479d89ed4f7db0d5b888874f8ac6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42159 } health_report { overall_health: HEALTHY } } }
I20260810 06:43:46.266281 14286 raft_consensus.cc:1261] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Refusing update from remote peer 72ae479d89ed4f7db0d5b888874f8ac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:46.266394 14501 raft_consensus.cc:1261] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Refusing update from remote peer 72ae479d89ed4f7db0d5b888874f8ac6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:43:46.267021 15020 consensus_queue.cc:1035] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c18629ed3134db7b12e2ef43873ecd5" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:43:46.267313 15032 consensus_queue.cc:1035] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "220f89114a334ea2ab42a24dece98c54" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:43:46.289026 14967 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:43:47.871104 14032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:47.871321 14032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:43:47.875051 14032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c51fa6d9f66d49d29489dd30b92c511a: Sending DeleteTablet for 3 replicas of tablet 4ac007e092d34715aefc3d24daa6a9b1
I20260810 06:43:47.875733 14258 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac007e092d34715aefc3d24daa6a9b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:47 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:52416
I20260810 06:43:47.875835 14696 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac007e092d34715aefc3d24daa6a9b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:47 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:56792
I20260810 06:43:47.875908 14473 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ac007e092d34715aefc3d24daa6a9b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:43:47 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:49614
I20260810 06:43:47.876427 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14152
I20260810 06:43:47.888128 15103 tablet_replica.cc:330] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: stopping tablet replica
I20260810 06:43:47.888468 15103 raft_consensus.cc:2229] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:47.888824 15103 raft_consensus.cc:2258] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:47.889964 15103 ts_tablet_manager.cc:1905] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:47.891157 15105 tablet_replica.cc:330] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: stopping tablet replica
I20260810 06:43:47.891469 15105 raft_consensus.cc:2229] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:43:47.891916 15105 raft_consensus.cc:2258] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:47.892941 15105 ts_tablet_manager.cc:1905] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:47.897223 15103 ts_tablet_manager.cc:1918] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:43:47.897763 15111 tablet_replica.cc:330] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: stopping tablet replica
I20260810 06:43:47.897996 15111 raft_consensus.cc:2229] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:43:47.898319 15111 raft_consensus.cc:2258] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:43:47.898612 15103 log.cc:1196] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/4ac007e092d34715aefc3d24daa6a9b1
I20260810 06:43:47.899390 15111 ts_tablet_manager.cc:1905] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:43:47.899659 15105 ts_tablet_manager.cc:1918] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:43:47.899832 15105 log.cc:1196] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/4ac007e092d34715aefc3d24daa6a9b1
I20260810 06:43:47.899842 15103 ts_tablet_manager.cc:1939] T 4ac007e092d34715aefc3d24daa6a9b1 P 9c18629ed3134db7b12e2ef43873ecd5: Deleting consensus metadata
I20260810 06:43:47.900264 15105 ts_tablet_manager.cc:1939] T 4ac007e092d34715aefc3d24daa6a9b1 P 72ae479d89ed4f7db0d5b888874f8ac6: Deleting consensus metadata
I20260810 06:43:47.900940 14004 catalog_manager.cc:4841] TS 9c18629ed3134db7b12e2ef43873ecd5 (127.10.242.1:43535): tablet 4ac007e092d34715aefc3d24daa6a9b1 (table test-table [id=2b5b67dfee6a4ddbb027d9ec88bc6d13]) successfully deleted
I20260810 06:43:47.901321 14004 catalog_manager.cc:4841] TS 72ae479d89ed4f7db0d5b888874f8ac6 (127.10.242.3:42159): tablet 4ac007e092d34715aefc3d24daa6a9b1 (table test-table [id=2b5b67dfee6a4ddbb027d9ec88bc6d13]) successfully deleted
I20260810 06:43:47.905002 15111 ts_tablet_manager.cc:1918] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:43:47.905114 15111 log.cc:1196] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/4ac007e092d34715aefc3d24daa6a9b1
I20260810 06:43:47.905468 15111 ts_tablet_manager.cc:1939] T 4ac007e092d34715aefc3d24daa6a9b1 P 220f89114a334ea2ab42a24dece98c54: Deleting consensus metadata
I20260810 06:43:47.906328 14004 catalog_manager.cc:4841] TS 220f89114a334ea2ab42a24dece98c54 (127.10.242.2:46529): tablet 4ac007e092d34715aefc3d24daa6a9b1 (table test-table [id=2b5b67dfee6a4ddbb027d9ec88bc6d13]) successfully deleted
Aug 10 06:43:47 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344225, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:43:47.912835 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:47.878284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.10.242.1:43535)
0810 06:43:47.898574 (+ 20290us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:47.898580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:47.898612 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:43:47.898807 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:47.898811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:47.898889 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:47.898986 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:47.900787 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:47.901892 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:47.903228 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:47.903856 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:47.903978 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:47.907911 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:47.907925 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:47.907943 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:47.907972 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:47.910581 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:47.911156 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:47.911162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:47.911164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:47.911300 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:47.911621 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:47.911624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:47.911626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:47.911935 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:47.912166 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:47.912434 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 06:43:47.912590 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20141,"thread_start_us":20008,"threads_started":1}
W20260810 06:43:47.913440 14359 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52438
W20260810 06:43:47.913975 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14152: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:47.914179 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 14152
I20260810 06:43:47.945593 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14373
Aug 10 06:43:47 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344225, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:43:47.959594 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:47.946313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49646 (local address 127.10.242.2:46529)
0810 06:43:47.946761 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:47.946766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:47.946782 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:43:47.946836 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:47.946839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:47.946898 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:47.946984 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:47.948143 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:47.948858 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:47.950066 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:47.950481 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:47.950570 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:47.954448 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:47.954463 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:47.954479 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:47.954518 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:47.957142 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:47.957782 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:47.957788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:47.957790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:47.957929 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:47.958362 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:47.958367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:47.958370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:47.958732 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:47.958958 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:47.959165 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 06:43:47.959343 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":227,"threads_started":1}
W20260810 06:43:47.960256 14579 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49646
W20260810 06:43:47.960853 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14373: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:47.961007 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 14373
I20260810 06:43:47.994024 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14591
Aug 10 06:43:47 dist-test-slave-zj8q krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344225, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:43:48.008193 15118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:47.994826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.10.242.3:42159)
0810 06:43:47.995349 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:47.995353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:47.995369 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:43:47.995423 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:47.995427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:47.995493 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:47.995582 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:47.996783 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:47.997507 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:47.998742 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:47.999140 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:47.999234 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:48.003176 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:48.003189 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:48.003207 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:48.003237 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:48.005818 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:48.006462 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:48.006470 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:48.006473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:48.006639 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:48.006966 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:48.006969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:48.006971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:48.007308 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:48.007523 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:48.007723 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 06:43:48.007882 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":261,"threads_started":1}
W20260810 06:43:48.008852 14813 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56816
W20260810 06:43:48.009433 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14591: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:48.009588 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 14591
I20260810 06:43:48.042018 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13973
I20260810 06:43:48.043176 14105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:48.166980 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13973
2026-08-10T06:43:48Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8391 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:48 dist-test-slave-zj8q krb5kdc[15123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:48 dist-test-slave-zj8q krb5kdc[15123](info): set up 2 sockets
Aug 10 06:43:48 dist-test-slave-zj8q krb5kdc[15123](info): commencing operation
krb5kdc: starting...
W20260810 06:43:49.395582 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:43:49 dist-test-slave-zj8q krb5kdc[15123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:43:49.583029 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:33131
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:33131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:49.900030 15149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:49.900296 15149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:49.900424 15149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:49.907379 15149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:49.907473 15149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:49.907547 15149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:49.907616 15149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:49.907680 15149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:49.917164 15149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:33131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:49.918423 15149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:49.919749 15149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:49.931128 15158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:49.931166 15162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:49.931102 15157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:49.931941 15149 server_base.cc:1034] running on GCE node
I20260810 06:43:49.932654 15149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:49.933916 15149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:49.935115 15149 hybrid_clock.cc:648] HybridClock initialized: now 1786344229935064 us; error 31 us; skew 500 ppm
I20260810 06:43:49.935431 15149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:49 dist-test-slave-zj8q krb5kdc[15123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344229, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:49.938374 15149 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:43:49.939761 15149 webserver.cc:460] Webserver started at http://127.10.242.62:45595/ using document root <none> and password file <none>
I20260810 06:43:49.940373 15149 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:49.940474 15149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:49.940738 15149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:49.943560 15149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "fa9ecacc2cea484d9d45781c3308296e"
format_stamp: "Formatted at 2026-08-10 06:43:49 on dist-test-slave-zj8q"
server_key: "a65695c09e91c6de4aa09075eeeb8d93"
server_key_iv: "278a43f3b07e7c6766c56dcf0fe46c31"
server_key_version: "encryptionkey@0"
I20260810 06:43:49.944254 15149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "fa9ecacc2cea484d9d45781c3308296e"
format_stamp: "Formatted at 2026-08-10 06:43:49 on dist-test-slave-zj8q"
server_key: "a65695c09e91c6de4aa09075eeeb8d93"
server_key_iv: "278a43f3b07e7c6766c56dcf0fe46c31"
server_key_version: "encryptionkey@0"
I20260810 06:43:49.949028 15149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:43:49.952205 15178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:49.953366 15149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 06:43:49.953521 15149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "fa9ecacc2cea484d9d45781c3308296e"
format_stamp: "Formatted at 2026-08-10 06:43:49 on dist-test-slave-zj8q"
server_key: "a65695c09e91c6de4aa09075eeeb8d93"
server_key_iv: "278a43f3b07e7c6766c56dcf0fe46c31"
server_key_version: "encryptionkey@0"
I20260810 06:43:49.953713 15149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:49.967605 15149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:49.971138 15149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:49.971386 15149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:49.988098 15149 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:33131
I20260810 06:43:49.988133 15267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:33131 every 8 connection(s)
I20260810 06:43:49.989333 15149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:49.992748 15269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:49.997987 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 15149
I20260810 06:43:49.998260 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:43:49.999039 11208 external_mini_cluster.cc:1428] Setting key 8c7cbfeab4bbecf4608aba5fc4c1a7b9
I20260810 06:43:50.003553 15269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e: Bootstrap starting.
I20260810 06:43:50.007025 15269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:50.008132 15269 log.cc:824] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:50.011063 15269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e: No bootstrap required, opened a new log
Aug 10 06:43:50 dist-test-slave-zj8q krb5kdc[15123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:50.021498 15269 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ecacc2cea484d9d45781c3308296e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33131 } }
I20260810 06:43:50.021936 15269 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:50.022053 15269 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa9ecacc2cea484d9d45781c3308296e, State: Initialized, Role: FOLLOWER
I20260810 06:43:50.022680 15269 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ecacc2cea484d9d45781c3308296e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33131 } }
I20260810 06:43:50.022908 15269 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:50.023056 15269 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:50.023216 15269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:50.023532 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:50.001103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.10.242.62:33131)
0810 06:43:50.001893 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:50.001904 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:50.001945 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:43:50.003622 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:50.003643 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:50.004014 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:50.004317 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:50.006141 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:50.007489 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:50.009099 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:50.009843 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:50.009965 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:50.015049 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:50.015060 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:50.015080 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:50.015126 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:50.019277 (+ 4151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:50.020131 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:50.020138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:50.020144 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:50.020327 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:50.020712 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:50.020715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:50.020717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:50.021255 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:50.021458 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:50.021994 (+ 536us) server_negotiation.cc:299] Negotiation successful
0810 06:43:50.022461 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":372,"threads_started":1}
I20260810 06:43:50.025548 15269 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ecacc2cea484d9d45781c3308296e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33131 } }
I20260810 06:43:50.026015 15269 leader_election.cc:302] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa9ecacc2cea484d9d45781c3308296e; no voters:
I20260810 06:43:50.026703 15269 leader_election.cc:288] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:50.026801 15278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:50.027489 15278 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [term 1 LEADER]: Becoming Leader. State: Replica: fa9ecacc2cea484d9d45781c3308296e, State: Running, Role: LEADER
I20260810 06:43:50.027979 15278 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ecacc2cea484d9d45781c3308296e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33131 } }
I20260810 06:43:50.028707 15269 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:50.030432 15280 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa9ecacc2cea484d9d45781c3308296e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ecacc2cea484d9d45781c3308296e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33131 } } }
I20260810 06:43:50.030778 15280 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:50.031586 15290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:50.031426 15281 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa9ecacc2cea484d9d45781c3308296e. Latest consensus state: current_term: 1 leader_uuid: "fa9ecacc2cea484d9d45781c3308296e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa9ecacc2cea484d9d45781c3308296e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33131 } } }
I20260810 06:43:50.031823 15281 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:50.036425 15290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:50.044499 15290 catalog_manager.cc:1347] Generated new cluster ID: 3d9382a427d641f49e79020ad1fee840
I20260810 06:43:50.044597 15290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:50.062278 15290 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:50.063536 15290 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:50.073823 15290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa9ecacc2cea484d9d45781c3308296e: Generated new TSK 0
I20260810 06:43:50.074878 15290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:43:50.134114 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33131
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:50.464001 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:50.464269 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:50.464407 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:50.471345 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:50.471452 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:50.471635 15317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:43:50.480858 15317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:50.482017 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:50.483264 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:50.495743 15326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:50.495828 15329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:50.495810 15327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:50.496819 15317 server_base.cc:1034] running on GCE node
I20260810 06:43:50.497385 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:50.498255 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:50.499472 15317 hybrid_clock.cc:648] HybridClock initialized: now 1786344230499410 us; error 44 us; skew 500 ppm
I20260810 06:43:50.499840 15317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:50 dist-test-slave-zj8q krb5kdc[15123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344230, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:50.502975 15317 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:43:50.504330 15317 webserver.cc:460] Webserver started at http://127.10.242.1:40011/ using document root <none> and password file <none>
I20260810 06:43:50.504966 15317 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:50.505090 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:50.505388 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:50.508237 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "0614d23a32e3445ab4b68f4b592a6ea2"
format_stamp: "Formatted at 2026-08-10 06:43:50 on dist-test-slave-zj8q"
server_key: "47d4e467ad9fd37e505616987eda82d6"
server_key_iv: "d86426b3a8c3a0d7a00c31246c2a4c68"
server_key_version: "encryptionkey@0"
I20260810 06:43:50.508985 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "0614d23a32e3445ab4b68f4b592a6ea2"
format_stamp: "Formatted at 2026-08-10 06:43:50 on dist-test-slave-zj8q"
server_key: "47d4e467ad9fd37e505616987eda82d6"
server_key_iv: "d86426b3a8c3a0d7a00c31246c2a4c68"
server_key_version: "encryptionkey@0"
I20260810 06:43:50.514241 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 06:43:50.517429 15342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:50.518693 15317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:50.518843 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "0614d23a32e3445ab4b68f4b592a6ea2"
format_stamp: "Formatted at 2026-08-10 06:43:50 on dist-test-slave-zj8q"
server_key: "47d4e467ad9fd37e505616987eda82d6"
server_key_iv: "d86426b3a8c3a0d7a00c31246c2a4c68"
server_key_version: "encryptionkey@0"
I20260810 06:43:50.519013 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:50.537801 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:50.541752 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:50.542001 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:50.542932 15317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:50.544183 15317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:50.544265 15317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:50.544378 15317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:50.544430 15317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:50.574607 15317 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:46371
I20260810 06:43:50.574673 15531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:46371 every 8 connection(s)
I20260810 06:43:50.575846 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:43:50.585206 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 15317
I20260810 06:43:50.585415 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:43:50.586040 11208 external_mini_cluster.cc:1428] Setting key 6dfece4d87b5f9547a7c3cb254f0a8fc
Aug 10 06:43:50 dist-test-slave-zj8q krb5kdc[15123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344230, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:50.595158 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:50.578300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:33563 (local address 127.10.242.62:33131)
0810 06:43:50.578737 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:50.578743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:50.579586 (+ 843us) server_negotiation.cc:407] Connection header received
0810 06:43:50.581405 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:50.581410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:50.581496 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:50.581593 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:50.583106 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:50.583820 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:50.585336 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:50.585785 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:50.585899 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:50.589772 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:50.589777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:50.589780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:50.589811 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:50.592215 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:50.593001 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:50.593004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:50.593007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:50.593133 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:50.593546 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:50.593549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:50.593551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:50.593798 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:50.593904 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:50.594733 (+ 829us) server_negotiation.cc:299] Negotiation successful
0810 06:43:50.594898 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":207,"threads_started":1}
I20260810 06:43:50.596187 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:50.578726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:33131 (local address 127.10.242.1:33563)
0810 06:43:50.579423 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:50.579467 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:50.581137 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:50.581837 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:50.581846 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:50.582378 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:50.582930 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:50.582948 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:50.583974 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:50.583978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:50.585196 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:50.585209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:50.586018 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:50.586026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:50.586297 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:50.586885 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:50.586910 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:50.589562 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:50.592346 (+ 2784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:50.592353 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:50.592368 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:50.592893 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:50.593228 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:50.593232 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:50.593235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:50.593456 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:50.593918 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:50.593925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:50.594362 (+ 437us) client_negotiation.cc:769] Sending connection context
0810 06:43:50.594634 (+ 272us) client_negotiation.cc:240] Negotiation successful
0810 06:43:50.595114 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":258,"threads_started":1}
I20260810 06:43:50.597599 15533 heartbeater.cc:347] Connected to a master server at 127.10.242.62:33131
I20260810 06:43:50.597880 15533 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:50.598577 15533 heartbeater.cc:511] Master 127.10.242.62:33131 requested a full tablet report, sending...
I20260810 06:43:50.600685 15204 ts_manager.cc:194] Registered new tserver with Master: 0614d23a32e3445ab4b68f4b592a6ea2 (127.10.242.1:46371)
I20260810 06:43:50.602720 15204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:33563
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:43:50.631482 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33131
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:50.947891 15544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:50.948143 15544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:50.948273 15544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:50.955163 15544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:50.955265 15544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:50.955425 15544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:43:50.964740 15544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:50.965978 15544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:50.967264 15544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:50.979714 15557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:50.979737 15558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:50.979655 15560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:50.980649 15544 server_base.cc:1034] running on GCE node
I20260810 06:43:50.981196 15544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:50.982074 15544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:50.983271 15544 hybrid_clock.cc:648] HybridClock initialized: now 1786344230983213 us; error 52 us; skew 500 ppm
I20260810 06:43:50.983569 15544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:50 dist-test-slave-zj8q krb5kdc[15123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344230, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:50.986611 15544 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:43:50.988169 15544 webserver.cc:460] Webserver started at http://127.10.242.2:38101/ using document root <none> and password file <none>
I20260810 06:43:50.988982 15544 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:50.989126 15544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:50.989478 15544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:50.993165 15544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "807d3d2db4784163b96c00fc2ad0caad"
format_stamp: "Formatted at 2026-08-10 06:43:50 on dist-test-slave-zj8q"
server_key: "9b8d62a365180a305a5e61cde26f00a6"
server_key_iv: "341a2e511569c83e0cd5179e2748f7b4"
server_key_version: "encryptionkey@0"
I20260810 06:43:50.994132 15544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "807d3d2db4784163b96c00fc2ad0caad"
format_stamp: "Formatted at 2026-08-10 06:43:50 on dist-test-slave-zj8q"
server_key: "9b8d62a365180a305a5e61cde26f00a6"
server_key_iv: "341a2e511569c83e0cd5179e2748f7b4"
server_key_version: "encryptionkey@0"
I20260810 06:43:51.000216 15544 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260810 06:43:51.004204 15573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:51.005568 15544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:43:51.005733 15544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "807d3d2db4784163b96c00fc2ad0caad"
format_stamp: "Formatted at 2026-08-10 06:43:50 on dist-test-slave-zj8q"
server_key: "9b8d62a365180a305a5e61cde26f00a6"
server_key_iv: "341a2e511569c83e0cd5179e2748f7b4"
server_key_version: "encryptionkey@0"
I20260810 06:43:51.005905 15544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:51.018846 15544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:51.022794 15544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:51.023051 15544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:51.023952 15544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:51.025210 15544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:51.025283 15544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:51.025396 15544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:51.025454 15544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:51.053660 15544 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:40741
I20260810 06:43:51.053691 15795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:40741 every 8 connection(s)
I20260810 06:43:51.054910 15544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:43:51.058542 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 15544
I20260810 06:43:51.058755 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:43:51.059330 11208 external_mini_cluster.cc:1428] Setting key b1a748894f32201a70744be7c8452a8c
Aug 10 06:43:51 dist-test-slave-zj8q krb5kdc[15123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344230, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:51.074707 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:51.057323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:56393 (local address 127.10.242.62:33131)
0810 06:43:51.057534 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:51.057541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:51.058875 (+ 1334us) server_negotiation.cc:407] Connection header received
0810 06:43:51.061058 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:51.061063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:51.061194 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:51.061282 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:51.062779 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:51.063802 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:51.065281 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:51.065866 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:51.065985 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:51.069221 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:51.069227 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:51.069230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:51.069261 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:51.071774 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:51.072556 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:51.072559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:51.072562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:51.072691 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:51.073107 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:51.073110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:51.073112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:51.073360 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:51.073474 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:51.074288 (+ 814us) server_negotiation.cc:299] Negotiation successful
0810 06:43:51.074461 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:43:51.076073 15802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:51.057808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:33131 (local address 127.10.242.2:56393)
0810 06:43:51.058660 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:51.058724 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:51.060779 (+ 2055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:51.061539 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:51.061547 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:51.062023 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:51.062607 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:51.062626 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:51.063931 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:51.063935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:51.065123 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:51.065135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:51.066043 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:51.066048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:51.066303 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:51.066894 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:51.066914 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:51.068996 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:51.071915 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:51.071923 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:51.071938 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:51.072458 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:51.072787 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:51.072792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:51.072795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:51.073020 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:51.073484 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:51.073491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:51.073865 (+ 374us) client_negotiation.cc:769] Sending connection context
0810 06:43:51.074131 (+ 266us) client_negotiation.cc:240] Negotiation successful
0810 06:43:51.074634 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":321,"threads_started":1}
I20260810 06:43:51.077605 15797 heartbeater.cc:347] Connected to a master server at 127.10.242.62:33131
I20260810 06:43:51.077939 15797 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:51.078874 15797 heartbeater.cc:511] Master 127.10.242.62:33131 requested a full tablet report, sending...
I20260810 06:43:51.080442 15204 ts_manager.cc:194] Registered new tserver with Master: 807d3d2db4784163b96c00fc2ad0caad (127.10.242.2:40741)
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
I20260810 06:43:51.081204 15204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:56393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:43:51.107120 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33131
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:51.424629 15807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:51.424889 15807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:51.425021 15807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:51.432008 15807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:51.432107 15807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:51.432260 15807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:43:51.441448 15807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33131
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:51.442648 15807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:51.443882 15807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:51.458626 15815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:51.459345 15816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:51.463613 15824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:51.464255 15807 server_base.cc:1034] running on GCE node
I20260810 06:43:51.465052 15807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:51.465987 15807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:51.467216 15807 hybrid_clock.cc:648] HybridClock initialized: now 1786344231467148 us; error 51 us; skew 500 ppm
I20260810 06:43:51.467528 15807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:51 dist-test-slave-zj8q krb5kdc[15123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344231, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:51.470592 15807 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:43:51.471971 15807 webserver.cc:460] Webserver started at http://127.10.242.3:40675/ using document root <none> and password file <none>
I20260810 06:43:51.472654 15807 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:51.472771 15807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:51.473064 15807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:51.475960 15807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "400d131fd82f4d96bb70551f94bf41b3"
format_stamp: "Formatted at 2026-08-10 06:43:51 on dist-test-slave-zj8q"
server_key: "67b0142a02248cddac6e08d217dd3a48"
server_key_iv: "77e4921899164be77b1e3740d3ba4b9a"
server_key_version: "encryptionkey@0"
I20260810 06:43:51.476725 15807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "400d131fd82f4d96bb70551f94bf41b3"
format_stamp: "Formatted at 2026-08-10 06:43:51 on dist-test-slave-zj8q"
server_key: "67b0142a02248cddac6e08d217dd3a48"
server_key_iv: "77e4921899164be77b1e3740d3ba4b9a"
server_key_version: "encryptionkey@0"
I20260810 06:43:51.481617 15807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:43:51.484953 15838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:51.486254 15807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:51.486403 15807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "400d131fd82f4d96bb70551f94bf41b3"
format_stamp: "Formatted at 2026-08-10 06:43:51 on dist-test-slave-zj8q"
server_key: "67b0142a02248cddac6e08d217dd3a48"
server_key_iv: "77e4921899164be77b1e3740d3ba4b9a"
server_key_version: "encryptionkey@0"
I20260810 06:43:51.486578 15807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:51.503677 15807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:51.507647 15807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:51.507898 15807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:51.508780 15807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:51.510178 15807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:51.510259 15807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:51.510380 15807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:51.510432 15807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:51.540724 15807 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:42527
I20260810 06:43:51.540772 16050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:42527 every 8 connection(s)
I20260810 06:43:51.541981 15807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:43:51.545847 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 15807
I20260810 06:43:51.546056 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:43:51.546603 11208 external_mini_cluster.cc:1428] Setting key 4d9a3e00280ea6f7864422f83df71062
Aug 10 06:43:51 dist-test-slave-zj8q krb5kdc[15123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344231, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:51.561410 15536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:51.544358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:58071 (local address 127.10.242.62:33131)
0810 06:43:51.544537 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:51.544542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:51.545767 (+ 1225us) server_negotiation.cc:407] Connection header received
0810 06:43:51.548094 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:51.548099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:51.548166 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:51.548249 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:51.549967 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:51.550769 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:51.552232 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:51.552609 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:51.552702 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:51.555879 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:51.555884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:51.555888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:51.555920 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:51.558595 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:51.559210 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:51.559213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:51.559215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:51.559332 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:51.559657 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:51.559660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:51.559662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:51.559910 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:51.560049 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:51.560714 (+ 665us) server_negotiation.cc:299] Negotiation successful
0810 06:43:51.560955 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:43:51.562335 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:51.544821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:33131 (local address 127.10.242.3:58071)
0810 06:43:51.545566 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:51.545631 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:51.547815 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:51.548519 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:51.548527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:51.549101 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:51.549816 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:51.549843 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:51.550890 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:51.550894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:51.552113 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:51.552138 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:51.552772 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:51.552777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:51.552960 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:51.553557 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:51.553584 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:51.555688 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:51.558708 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:51.558715 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:51.558726 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:51.559106 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:51.559407 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:51.559410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:51.559412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:51.559577 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:51.560045 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:51.560053 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:51.560358 (+ 305us) client_negotiation.cc:769] Sending connection context
0810 06:43:51.560598 (+ 240us) client_negotiation.cc:240] Negotiation successful
0810 06:43:51.561104 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":277,"threads_started":1}
I20260810 06:43:51.563563 16051 heartbeater.cc:347] Connected to a master server at 127.10.242.62:33131
I20260810 06:43:51.563849 16051 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:51.564539 16051 heartbeater.cc:511] Master 127.10.242.62:33131 requested a full tablet report, sending...
I20260810 06:43:51.565878 15204 ts_manager.cc:194] Registered new tserver with Master: 400d131fd82f4d96bb70551f94bf41b3 (127.10.242.3:42527)
I20260810 06:43:51.566696 15204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:58071
I20260810 06:43:51.573765 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:43:51.605698 15533 heartbeater.cc:503] Master 127.10.242.62:33131 was elected leader, sending a full tablet report...
W20260810 06:43:51.608469 15536 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:43:51.606021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49426 (local address 127.10.242.62:33131)
0810 06:43:51.606315 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:51.606323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:51.606689 (+ 366us) server_negotiation.cc:407] Connection header received
0810 06:43:51.606836 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:51.606841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:51.606922 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:51.607031 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:51.608221 (+ 1190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49426: BlockingRecv error: recv got EOF from 127.0.0.1:49426 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:43:51.611548 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15317
W20260810 06:43:51.614331 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15317: Not authorized: Client connection negotiation failed: client connection to 127.10.242.1:46371: server requires authentication, but client does not have Kerberos credentials available
I20260810 06:43:51.614513 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 15317
I20260810 06:43:51.639500 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15544
W20260810 06:43:51.642043 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15544: Not authorized: Client connection negotiation failed: client connection to 127.10.242.2:40741: server requires authentication, but client does not have Kerberos credentials available
I20260810 06:43:51.642248 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 15544
I20260810 06:43:51.664459 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15807
W20260810 06:43:51.667331 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15807: Not authorized: Client connection negotiation failed: client connection to 127.10.242.3:42527: server requires authentication, but client does not have Kerberos credentials available
I20260810 06:43:51.667464 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 15807
I20260810 06:43:51.683446 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15149
I20260810 06:43:51.684600 15259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:51.788091 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 15149
2026-08-10T06:43:51Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3618 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:51 dist-test-slave-zj8q krb5kdc[16084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:51 dist-test-slave-zj8q krb5kdc[16084](info): set up 2 sockets
Aug 10 06:43:51 dist-test-slave-zj8q krb5kdc[16084](info): commencing operation
krb5kdc: starting...
W20260810 06:43:53.007072 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:43:53 dist-test-slave-zj8q krb5kdc[16084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:43:53.203505 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:43149
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:43149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:53.520550 16100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:53.520861 16100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:53.520992 16100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:53.528086 16100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:53.528208 16100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:53.528287 16100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:53.528363 16100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:53.528425 16100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:53.538044 16100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:43149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:43149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:53.539394 16100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:53.540735 16100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:53.552264 16110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:53.552313 16114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:53.552346 16117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:53.553102 16100 server_base.cc:1034] running on GCE node
I20260810 06:43:53.553866 16100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:53.555318 16100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:53.556538 16100 hybrid_clock.cc:648] HybridClock initialized: now 1786344233556500 us; error 29 us; skew 500 ppm
I20260810 06:43:53.556879 16100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:53 dist-test-slave-zj8q krb5kdc[16084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344233, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:53.563746 16100 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:43:53.565169 16100 webserver.cc:460] Webserver started at http://127.10.242.62:43643/ using document root <none> and password file <none>
I20260810 06:43:53.565867 16100 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:53.565987 16100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:53.566339 16100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:53.569306 16100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "94316819222045b68213ff3cb19a00a7"
format_stamp: "Formatted at 2026-08-10 06:43:53 on dist-test-slave-zj8q"
server_key: "265efcee934265a0f5b91d92d7063130"
server_key_iv: "a512aa8ee60d06744d58eab6e8160faa"
server_key_version: "encryptionkey@0"
I20260810 06:43:53.570096 16100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "94316819222045b68213ff3cb19a00a7"
format_stamp: "Formatted at 2026-08-10 06:43:53 on dist-test-slave-zj8q"
server_key: "265efcee934265a0f5b91d92d7063130"
server_key_iv: "a512aa8ee60d06744d58eab6e8160faa"
server_key_version: "encryptionkey@0"
I20260810 06:43:53.575109 16100 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:43:53.578339 16126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:53.579562 16100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:43:53.579735 16100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "94316819222045b68213ff3cb19a00a7"
format_stamp: "Formatted at 2026-08-10 06:43:53 on dist-test-slave-zj8q"
server_key: "265efcee934265a0f5b91d92d7063130"
server_key_iv: "a512aa8ee60d06744d58eab6e8160faa"
server_key_version: "encryptionkey@0"
I20260810 06:43:53.579914 16100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:53.610296 16100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:53.614426 16100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:53.614688 16100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:53.631273 16100 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:43149
I20260810 06:43:53.631323 16207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:43149 every 8 connection(s)
I20260810 06:43:53.632508 16100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:53.636117 16209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:53.640595 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16100
I20260810 06:43:53.640785 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:43:53.641403 11208 external_mini_cluster.cc:1428] Setting key 0c74d6c4b9684f8adf9337b8fd2c1b1a
I20260810 06:43:53.647791 16209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7: Bootstrap starting.
I20260810 06:43:53.651341 16209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:53.652329 16209 log.cc:824] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:53.655078 16209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7: No bootstrap required, opened a new log
Aug 10 06:43:53 dist-test-slave-zj8q krb5kdc[16084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:53.663599 16209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94316819222045b68213ff3cb19a00a7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43149 } }
I20260810 06:43:53.663961 16209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:53.664074 16209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94316819222045b68213ff3cb19a00a7, State: Initialized, Role: FOLLOWER
I20260810 06:43:53.664744 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:53.643158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60086 (local address 127.10.242.62:43149)
0810 06:43:53.644072 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:53.644087 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:53.644141 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:43:53.646579 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:53.646608 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:53.647058 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:53.647482 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:53.648883 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:53.650905 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:53.652204 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:53.653188 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:53.653316 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:53.657683 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:53.657691 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:53.657707 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:53.657750 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:53.660685 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:53.661470 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:53.661476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:53.661482 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:53.661657 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:53.662049 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:53.662052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:53.662054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:53.662671 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:53.662880 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:53.663308 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 06:43:53.663614 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":376,"threads_started":1}
I20260810 06:43:53.664659 16209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94316819222045b68213ff3cb19a00a7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43149 } }
I20260810 06:43:53.664887 16209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:53.665053 16209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:53.665230 16209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:53.667986 16209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94316819222045b68213ff3cb19a00a7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43149 } }
I20260810 06:43:53.668596 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:43:53.668512 16209 leader_election.cc:302] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94316819222045b68213ff3cb19a00a7; no voters:
I20260810 06:43:53.669082 16209 leader_election.cc:288] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:53.669214 16218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:53.670401 16218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [term 1 LEADER]: Becoming Leader. State: Replica: 94316819222045b68213ff3cb19a00a7, State: Running, Role: LEADER
I20260810 06:43:53.671082 16209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:53.670987 16218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94316819222045b68213ff3cb19a00a7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43149 } }
I20260810 06:43:53.675707 16219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94316819222045b68213ff3cb19a00a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94316819222045b68213ff3cb19a00a7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43149 } } }
I20260810 06:43:53.676188 16219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:53.676857 16220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94316819222045b68213ff3cb19a00a7. Latest consensus state: current_term: 1 leader_uuid: "94316819222045b68213ff3cb19a00a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94316819222045b68213ff3cb19a00a7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43149 } } }
I20260810 06:43:53.677114 16220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:53.679466 16229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:53.684731 16229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:53.702430 16229 catalog_manager.cc:1347] Generated new cluster ID: 081f206f62e0410fb37512c067296d15
I20260810 06:43:53.702605 16229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:53.729249 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:53.719972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.10.242.62:43149)
0810 06:43:53.720168 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:53.720174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:53.720742 (+ 568us) server_negotiation.cc:407] Connection header received
0810 06:43:53.720949 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:53.720955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:53.721031 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:53.721128 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:53.722487 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:53.723571 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:53.725476 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:53.726128 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:53.726270 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:53.727428 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:53.727436 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:53.727440 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:43:53.728273 (+ 833us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:53.728343 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:43:53.728380 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:53.728482 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:53.728638 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 06:43:53.728735 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:43:53.736294 16229 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:53.737548 16229 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:53.754026 16229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94316819222045b68213ff3cb19a00a7: Generated new TSK 0
I20260810 06:43:53.754885 16229 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:43:53.761714 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16100
I20260810 06:43:53.762908 16200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:53.854185 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16100
2026-08-10T06:43:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2057 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:53 dist-test-slave-zj8q krb5kdc[16264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:53 dist-test-slave-zj8q krb5kdc[16264](info): set up 2 sockets
Aug 10 06:43:53 dist-test-slave-zj8q krb5kdc[16264](info): commencing operation
krb5kdc: starting...
W20260810 06:43:55.065680 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.016s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:43:55 dist-test-slave-zj8q krb5kdc[16264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:43:55.267244 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:41845
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:41845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:55.586676 16290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:55.586949 16290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:55.587054 16290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:55.594017 16290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:55.594120 16290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:55.594203 16290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:55.594269 16290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:55.594312 16290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:55.603782 16290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:41845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:41845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:55.605010 16290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:55.606326 16290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:55.617669 16296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:55.617616 16297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:55.617583 16299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:55.618916 16290 server_base.cc:1034] running on GCE node
I20260810 06:43:55.619678 16290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:55.621130 16290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:55.622330 16290 hybrid_clock.cc:648] HybridClock initialized: now 1786344235622280 us; error 33 us; skew 500 ppm
I20260810 06:43:55.622673 16290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:55 dist-test-slave-zj8q krb5kdc[16264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344235, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:55.625737 16290 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:43:55.627188 16290 webserver.cc:460] Webserver started at http://127.10.242.62:43549/ using document root <none> and password file <none>
I20260810 06:43:55.627871 16290 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:55.628005 16290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:55.628298 16290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:55.631276 16290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "a3913552d9c14d13975d1418bbfa0a6e"
format_stamp: "Formatted at 2026-08-10 06:43:55 on dist-test-slave-zj8q"
server_key: "e93dcb6e4afe279b2b125f7f7ed852c2"
server_key_iv: "8300c96c68486f970584ce4f839e78be"
server_key_version: "encryptionkey@0"
I20260810 06:43:55.632035 16290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "a3913552d9c14d13975d1418bbfa0a6e"
format_stamp: "Formatted at 2026-08-10 06:43:55 on dist-test-slave-zj8q"
server_key: "e93dcb6e4afe279b2b125f7f7ed852c2"
server_key_iv: "8300c96c68486f970584ce4f839e78be"
server_key_version: "encryptionkey@0"
I20260810 06:43:55.637066 16290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:43:55.640360 16312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:55.641582 16290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 06:43:55.641753 16290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "a3913552d9c14d13975d1418bbfa0a6e"
format_stamp: "Formatted at 2026-08-10 06:43:55 on dist-test-slave-zj8q"
server_key: "e93dcb6e4afe279b2b125f7f7ed852c2"
server_key_iv: "8300c96c68486f970584ce4f839e78be"
server_key_version: "encryptionkey@0"
I20260810 06:43:55.641929 16290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:55.664132 16290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:55.668304 16290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:55.668566 16290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:55.685595 16290 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:41845
I20260810 06:43:55.685650 16393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:41845 every 8 connection(s)
I20260810 06:43:55.686926 16290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:55.690605 16396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:55.695180 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16290
I20260810 06:43:55.695387 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:43:55.696020 11208 external_mini_cluster.cc:1428] Setting key c317e14460d40db10138755554f278e8
I20260810 06:43:55.701020 16396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e: Bootstrap starting.
I20260810 06:43:55.704574 16396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:55.705823 16396 log.cc:824] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:43:55.709209 16396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e: No bootstrap required, opened a new log
Aug 10 06:43:55 dist-test-slave-zj8q krb5kdc[16264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:55.717860 16396 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3913552d9c14d13975d1418bbfa0a6e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41845 } }
I20260810 06:43:55.718230 16396 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:55.718343 16396 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3913552d9c14d13975d1418bbfa0a6e, State: Initialized, Role: FOLLOWER
I20260810 06:43:55.719018 16400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:55.697890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.10.242.62:41845)
0810 06:43:55.698780 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:55.698792 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:55.698837 (+ 45us) server_negotiation.cc:407] Connection header received
0810 06:43:55.700437 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:55.700460 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:55.700837 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:55.701170 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:55.702900 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:55.704202 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:55.705941 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:55.706928 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:55.707122 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:55.711754 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:55.711762 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:55.711779 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:55.711815 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:55.714912 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:55.715688 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:55.715694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:55.715700 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:55.715880 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:55.716294 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:55.716297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:55.716299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:55.716817 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:55.717048 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:55.717483 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 06:43:55.717832 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":393,"threads_started":1}
I20260810 06:43:55.718914 16396 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3913552d9c14d13975d1418bbfa0a6e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41845 } }
I20260810 06:43:55.719173 16396 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:55.719328 16396 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:55.719527 16396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:55.722409 16396 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3913552d9c14d13975d1418bbfa0a6e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41845 } }
I20260810 06:43:55.722870 16396 leader_election.cc:302] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3913552d9c14d13975d1418bbfa0a6e; no voters:
I20260810 06:43:55.723438 16396 leader_election.cc:288] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:55.723599 16402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:55.724206 16402 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [term 1 LEADER]: Becoming Leader. State: Replica: a3913552d9c14d13975d1418bbfa0a6e, State: Running, Role: LEADER
I20260810 06:43:55.724763 16402 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3913552d9c14d13975d1418bbfa0a6e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41845 } }
I20260810 06:43:55.726680 16402 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3913552d9c14d13975d1418bbfa0a6e. Latest consensus state: current_term: 1 leader_uuid: "a3913552d9c14d13975d1418bbfa0a6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3913552d9c14d13975d1418bbfa0a6e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41845 } } }
I20260810 06:43:55.726950 16402 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:55.727376 16405 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3913552d9c14d13975d1418bbfa0a6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3913552d9c14d13975d1418bbfa0a6e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41845 } } }
I20260810 06:43:55.728111 16396 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:55.728655 16405 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:55.730036 16407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:55.734800 16407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:55.742908 16407 catalog_manager.cc:1347] Generated new cluster ID: 18341f4b91fd4cc2bb941d9d5af5496a
I20260810 06:43:55.743014 16407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:55.762850 16407 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:55.764619 16407 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:55.772759 16407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3913552d9c14d13975d1418bbfa0a6e: Generated new TSK 0
I20260810 06:43:55.773761 16407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:43:55.821202 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:41845
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:56.146392 16448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:56.146695 16448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:56.146850 16448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:56.153841 16448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:56.153942 16448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:56.154095 16448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:43:56.163401 16448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:41845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:56.164681 16448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:56.166003 16448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:56.178440 16462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:56.178483 16458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:56.178507 16460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:56.179333 16448 server_base.cc:1034] running on GCE node
I20260810 06:43:56.179889 16448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:56.180934 16448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:56.182138 16448 hybrid_clock.cc:648] HybridClock initialized: now 1786344236182083 us; error 39 us; skew 500 ppm
I20260810 06:43:56.182528 16448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:56 dist-test-slave-zj8q krb5kdc[16264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344236, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:56.185694 16448 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:43:56.187060 16448 webserver.cc:460] Webserver started at http://127.10.242.1:43295/ using document root <none> and password file <none>
I20260810 06:43:56.187700 16448 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:56.187803 16448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:56.188054 16448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:56.190915 16448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "9ad32c241ebc4b26bae9674212b9dbba"
format_stamp: "Formatted at 2026-08-10 06:43:56 on dist-test-slave-zj8q"
server_key: "2de8923d2088d84cb0df289f6c912013"
server_key_iv: "135b8ca443a2b25180edec0aaa089b11"
server_key_version: "encryptionkey@0"
I20260810 06:43:56.191642 16448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "9ad32c241ebc4b26bae9674212b9dbba"
format_stamp: "Formatted at 2026-08-10 06:43:56 on dist-test-slave-zj8q"
server_key: "2de8923d2088d84cb0df289f6c912013"
server_key_iv: "135b8ca443a2b25180edec0aaa089b11"
server_key_version: "encryptionkey@0"
I20260810 06:43:56.196436 16448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:43:56.199689 16471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:56.200915 16448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:43:56.201050 16448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "9ad32c241ebc4b26bae9674212b9dbba"
format_stamp: "Formatted at 2026-08-10 06:43:56 on dist-test-slave-zj8q"
server_key: "2de8923d2088d84cb0df289f6c912013"
server_key_iv: "135b8ca443a2b25180edec0aaa089b11"
server_key_version: "encryptionkey@0"
I20260810 06:43:56.201215 16448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:56.217756 16448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:56.221783 16448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:56.222020 16448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:56.222932 16448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:56.224175 16448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:56.224247 16448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:56.224337 16448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:56.224380 16448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:56.254004 16448 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:42663
I20260810 06:43:56.254051 16663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:42663 every 8 connection(s)
I20260810 06:43:56.255270 16448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:43:56.261709 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16448
I20260810 06:43:56.261888 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:43:56.262555 11208 external_mini_cluster.cc:1428] Setting key 07c2b8170aa2f2669af502b546bb0a39
Aug 10 06:43:56 dist-test-slave-zj8q krb5kdc[16264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344236, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:56.274109 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:56.257632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:42773 (local address 127.10.242.62:41845)
0810 06:43:56.258069 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:56.258074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:56.259061 (+ 987us) server_negotiation.cc:407] Connection header received
0810 06:43:56.260827 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:56.260833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:56.260910 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:56.261007 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:56.262566 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:56.263452 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:56.265080 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:56.265510 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:56.265649 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:56.268905 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:56.268910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:56.268913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:56.268944 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:56.271324 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:56.271998 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:56.272003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:56.272005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:56.272116 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:56.272517 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:56.272521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:56.272523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:56.272791 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:56.272943 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:56.273671 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 06:43:56.273839 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":210,"threads_started":1}
I20260810 06:43:56.275386 16672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:56.258221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41845 (local address 127.10.242.1:42773)
0810 06:43:56.258905 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:56.258946 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:56.260547 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:56.261249 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:56.261257 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:56.261770 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:56.262400 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:56.262420 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:56.263598 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:56.263602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:56.264941 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:56.264954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:56.265770 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:56.265777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:56.265992 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:56.266610 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:56.266638 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:56.268701 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:56.271451 (+ 2750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:56.271459 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:56.271475 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:56.271872 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:56.272215 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:56.272220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:56.272224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:56.272402 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:56.272913 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:56.272921 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:56.273297 (+ 376us) client_negotiation.cc:769] Sending connection context
0810 06:43:56.273570 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 06:43:56.274060 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":260,"threads_started":1}
I20260810 06:43:56.276834 16668 heartbeater.cc:347] Connected to a master server at 127.10.242.62:41845
I20260810 06:43:56.277153 16668 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:56.277988 16668 heartbeater.cc:511] Master 127.10.242.62:41845 requested a full tablet report, sending...
I20260810 06:43:56.280539 16347 ts_manager.cc:194] Registered new tserver with Master: 9ad32c241ebc4b26bae9674212b9dbba (127.10.242.1:42663)
I20260810 06:43:56.282135 16347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:42773
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:43:56.311041 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:41845
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:56.625602 16678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:56.625885 16678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:56.626015 16678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:56.633139 16678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:56.633251 16678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:56.633400 16678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:43:56.642702 16678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:41845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:56.643893 16678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:56.645183 16678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:56.657826 16684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:56.657868 16689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:56.657902 16685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:56.658661 16678 server_base.cc:1034] running on GCE node
I20260810 06:43:56.659200 16678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:56.660027 16678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:56.661219 16678 hybrid_clock.cc:648] HybridClock initialized: now 1786344236661178 us; error 32 us; skew 500 ppm
I20260810 06:43:56.661515 16678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:56 dist-test-slave-zj8q krb5kdc[16264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344236, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:56.664561 16678 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:43:56.666028 16678 webserver.cc:460] Webserver started at http://127.10.242.2:33149/ using document root <none> and password file <none>
I20260810 06:43:56.666702 16678 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:56.666807 16678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:56.667058 16678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:56.669834 16678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "e6e2848c7cb74cbba8da892731d64877"
format_stamp: "Formatted at 2026-08-10 06:43:56 on dist-test-slave-zj8q"
server_key: "340846087c04d8670163d3c35fe8d8ea"
server_key_iv: "a00e77aadbe1e5a9475ca9ec5e561095"
server_key_version: "encryptionkey@0"
I20260810 06:43:56.670606 16678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "e6e2848c7cb74cbba8da892731d64877"
format_stamp: "Formatted at 2026-08-10 06:43:56 on dist-test-slave-zj8q"
server_key: "340846087c04d8670163d3c35fe8d8ea"
server_key_iv: "a00e77aadbe1e5a9475ca9ec5e561095"
server_key_version: "encryptionkey@0"
I20260810 06:43:56.675446 16678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 06:43:56.678664 16699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:56.679867 16678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:43:56.680016 16678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "e6e2848c7cb74cbba8da892731d64877"
format_stamp: "Formatted at 2026-08-10 06:43:56 on dist-test-slave-zj8q"
server_key: "340846087c04d8670163d3c35fe8d8ea"
server_key_iv: "a00e77aadbe1e5a9475ca9ec5e561095"
server_key_version: "encryptionkey@0"
I20260810 06:43:56.680189 16678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:56.697350 16678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:56.701331 16678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:56.701594 16678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:56.702521 16678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:56.703790 16678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:56.703868 16678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:56.703994 16678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:56.704048 16678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:56.733938 16678 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:39819
I20260810 06:43:56.733984 16884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:39819 every 8 connection(s)
I20260810 06:43:56.735193 16678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:43:56.738564 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16678
I20260810 06:43:56.738744 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:43:56.739202 11208 external_mini_cluster.cc:1428] Setting key 1e226c22562ef24d2b49f9e975c2f2c0
Aug 10 06:43:56 dist-test-slave-zj8q krb5kdc[16264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344236, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:56.755012 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:56.737620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:39297 (local address 127.10.242.62:41845)
0810 06:43:56.737809 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:56.737813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:56.739071 (+ 1258us) server_negotiation.cc:407] Connection header received
0810 06:43:56.741196 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:56.741201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:56.741286 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:56.741385 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:56.742851 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:56.743851 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:56.745436 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:56.746046 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:56.746208 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:56.749613 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:56.749619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:56.749629 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:56.749662 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:56.752093 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:56.752771 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:56.752775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:56.752777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:56.752906 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:56.753328 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:56.753332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:56.753334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:56.753583 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:56.753739 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:56.754441 (+ 702us) server_negotiation.cc:299] Negotiation successful
0810 06:43:56.754681 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:43:56.755965 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:56.738067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41845 (local address 127.10.242.2:39297)
0810 06:43:56.738904 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:56.738958 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:56.740916 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:56.741576 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:56.741584 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:56.742072 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:56.742683 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:56.742701 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:56.743978 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:56.743981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:56.745307 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:56.745319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:56.746307 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:56.746312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:56.746515 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:56.747137 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:56.747160 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:56.749406 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:56.752225 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:56.752234 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:56.752250 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:56.752651 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:56.753011 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:56.753016 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:56.753020 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:56.753216 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:56.753714 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:56.753722 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:56.754041 (+ 319us) client_negotiation.cc:769] Sending connection context
0810 06:43:56.754325 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 06:43:56.754826 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":309,"threads_started":1}
I20260810 06:43:56.757371 16886 heartbeater.cc:347] Connected to a master server at 127.10.242.62:41845
I20260810 06:43:56.757658 16886 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:56.758404 16886 heartbeater.cc:511] Master 127.10.242.62:41845 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
I20260810 06:43:56.759773 16347 ts_manager.cc:194] Registered new tserver with Master: e6e2848c7cb74cbba8da892731d64877 (127.10.242.2:39819)
I20260810 06:43:56.760540 16347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:39297
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:43:56.785398 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:41845
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:57.106845 16893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:57.107127 16893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:57.107268 16893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:57.114704 16893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:57.114830 16893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:57.115006 16893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:43:57.124833 16893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:41845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:57.126261 16893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:57.127723 16893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:57.140802 16899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:57.140837 16900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:57.140961 16902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:57.141948 16893 server_base.cc:1034] running on GCE node
I20260810 06:43:57.142570 16893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:57.143478 16893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:57.144668 16893 hybrid_clock.cc:648] HybridClock initialized: now 1786344237144625 us; error 30 us; skew 500 ppm
I20260810 06:43:57.144966 16893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[16264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344237, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:57.148052 16893 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:43:57.149435 16893 webserver.cc:460] Webserver started at http://127.10.242.3:34951/ using document root <none> and password file <none>
I20260810 06:43:57.150068 16893 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:57.150207 16893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:57.150501 16893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:57.153290 16893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "8945c0ee14af42dd990c6759bd6668e6"
format_stamp: "Formatted at 2026-08-10 06:43:57 on dist-test-slave-zj8q"
server_key: "330ff0a7acf42b6f346fcdf79f2f9e05"
server_key_iv: "6e407f64f2a647a6967ae0ad39a97267"
server_key_version: "encryptionkey@0"
I20260810 06:43:57.154037 16893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "8945c0ee14af42dd990c6759bd6668e6"
format_stamp: "Formatted at 2026-08-10 06:43:57 on dist-test-slave-zj8q"
server_key: "330ff0a7acf42b6f346fcdf79f2f9e05"
server_key_iv: "6e407f64f2a647a6967ae0ad39a97267"
server_key_version: "encryptionkey@0"
I20260810 06:43:57.159022 16893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:43:57.162323 16917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:57.163545 16893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:43:57.163697 16893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "8945c0ee14af42dd990c6759bd6668e6"
format_stamp: "Formatted at 2026-08-10 06:43:57 on dist-test-slave-zj8q"
server_key: "330ff0a7acf42b6f346fcdf79f2f9e05"
server_key_iv: "6e407f64f2a647a6967ae0ad39a97267"
server_key_version: "encryptionkey@0"
I20260810 06:43:57.163873 16893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:57.180859 16893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:57.185124 16893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:57.185447 16893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:57.186719 16893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:57.188194 16893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:57.188268 16893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:57.188382 16893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:57.188436 16893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:57.219022 16893 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:38539
I20260810 06:43:57.219062 17098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:38539 every 8 connection(s)
I20260810 06:43:57.220355 16893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:43:57.227505 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16893
I20260810 06:43:57.227710 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:43:57.228240 11208 external_mini_cluster.cc:1428] Setting key 1925da8d86de01451e45e7ddb505b42f
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[16264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344237, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:57.240322 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:57.222950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:56649 (local address 127.10.242.62:41845)
0810 06:43:57.223150 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:57.223155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:57.224294 (+ 1139us) server_negotiation.cc:407] Connection header received
0810 06:43:57.226076 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:57.226083 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:57.226199 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:57.226289 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:57.228086 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.229036 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.231060 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.231614 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.231711 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:57.235044 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:57.235049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:57.235053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:57.235085 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:57.237588 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.238353 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.238357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.238359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.238476 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.238883 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.238886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.238888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.239149 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:57.239268 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:57.239888 (+ 620us) server_negotiation.cc:299] Negotiation successful
0810 06:43:57.240056 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:43:57.241277 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:57.223432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41845 (local address 127.10.242.3:56649)
0810 06:43:57.224138 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0810 06:43:57.224177 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:43:57.225794 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:43:57.226549 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:43:57.226557 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:43:57.227038 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:43:57.227850 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:57.227905 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.229200 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.229206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:57.230932 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:43:57.230946 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.231812 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.231817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:43:57.232073 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:57.232661 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:43:57.232683 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:43:57.234850 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:43:57.237757 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:57.237764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:57.237776 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:57.238259 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:57.238596 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:43:57.238599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:43:57.238602 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:43:57.238797 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:43:57.239254 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:43:57.239260 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:43:57.239568 (+ 308us) client_negotiation.cc:769] Sending connection context
0810 06:43:57.239816 (+ 248us) client_negotiation.cc:240] Negotiation successful
0810 06:43:57.240195 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":264,"threads_started":1}
I20260810 06:43:57.242955 17099 heartbeater.cc:347] Connected to a master server at 127.10.242.62:41845
I20260810 06:43:57.243252 17099 heartbeater.cc:464] Registering TS with master...
I20260810 06:43:57.244074 17099 heartbeater.cc:511] Master 127.10.242.62:41845 requested a full tablet report, sending...
I20260810 06:43:57.245464 16344 ts_manager.cc:194] Registered new tserver with Master: 8945c0ee14af42dd990c6759bd6668e6 (127.10.242.3:38539)
I20260810 06:43:57.246400 16344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:56649
I20260810 06:43:57.255990 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:43:57.285724 16668 heartbeater.cc:503] Master 127.10.242.62:41845 was elected leader, sending a full tablet report...
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[16264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344237, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[16264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344237, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:57.309283 16670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:57.295472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.10.242.62:41845)
0810 06:43:57.295692 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:57.295700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:57.296068 (+ 368us) server_negotiation.cc:407] Connection header received
0810 06:43:57.296206 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:57.296209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:57.296275 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:57.296357 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:57.297727 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.298552 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.299746 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.300193 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.300296 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:57.304211 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:57.304216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:57.304220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:57.304255 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:57.307018 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.307629 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.307633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.307635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.307765 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.308053 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.308056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.308058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.308393 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:57.308617 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:57.308809 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 06:43:57.308980 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 06:43:57.309922 16344 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36412
I20260810 06:43:57.311717 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16448
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[16264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344237, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:43:57.327054 17115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:57.312781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.10.242.1:42663)
0810 06:43:57.313414 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:57.313421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:57.313445 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:43:57.313500 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:57.313507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:57.313721 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:57.313845 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:57.315050 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.316068 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.317245 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.317813 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.317917 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:57.321692 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:57.321704 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:57.321720 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:57.321755 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:57.324601 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.325221 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.325228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.325231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.325353 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.325653 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.325656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.325658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.326001 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:57.326278 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:57.326483 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 06:43:57.326684 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":341,"threads_started":1}
W20260810 06:43:57.328363 16655 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32862
W20260810 06:43:57.329418 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16448: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:57.329638 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16448
I20260810 06:43:57.355435 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16678
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[16264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344237, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:43:57.371780 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:57.356438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37704 (local address 127.10.242.2:39819)
0810 06:43:57.356973 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:57.356980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:57.357006 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:43:57.357096 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:57.357103 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:57.357313 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:57.357467 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:57.358920 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.359961 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.361245 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.361790 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.361911 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:57.366072 (+ 4161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:57.366082 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:57.366099 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:57.366134 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:57.369013 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.369648 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.369655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.369658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.369787 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.370177 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.370182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.370186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.370545 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:57.370838 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:57.371178 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 06:43:57.371441 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":269,"threads_started":1}
W20260810 06:43:57.373060 16876 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37704
W20260810 06:43:57.374109 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16678: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:57.374375 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16678
I20260810 06:43:57.397363 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16893
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[16264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344237, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:43:57.413172 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:57.398405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.10.242.3:38539)
0810 06:43:57.399069 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:57.399077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:57.399108 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:43:57.399178 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:57.399186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:57.399398 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:57.399529 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:57.400794 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.401832 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.403124 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:57.403697 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:57.403791 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:57.407979 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:57.407990 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:57.408006 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:57.408039 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:57.410804 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.411416 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.411422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.411425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.411539 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:57.411819 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:57.411822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:57.411825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:57.412160 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:57.412389 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:57.412608 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 06:43:57.412814 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":350,"threads_started":1}
W20260810 06:43:57.414433 17093 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37832
W20260810 06:43:57.415525 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16893: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:43:57.415719 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16893
I20260810 06:43:57.441329 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16290
I20260810 06:43:57.442543 16387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:43:57.545159 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16290
2026-08-10T06:43:57Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3692 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[17125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[17125](info): set up 2 sockets
Aug 10 06:43:57 dist-test-slave-zj8q krb5kdc[17125](info): commencing operation
krb5kdc: starting...
W20260810 06:43:58.769208 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:43:58 dist-test-slave-zj8q krb5kdc[17125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:43:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:43:58.971554 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:38713
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:38203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:38713
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:59.291275 17141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:59.291651 17141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:59.291796 17141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:59.298983 17141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:43:59.299077 17141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:59.299129 17141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:59.299185 17141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:43:59.299227 17141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:43:59.308796 17141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:38203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:38713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:38713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:59.310017 17141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:59.311264 17141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:59.322742 17146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:59.322662 17155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:59.322780 17151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:59.324340 17141 server_base.cc:1034] running on GCE node
I20260810 06:43:59.325088 17141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:59.326505 17141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:59.327761 17141 hybrid_clock.cc:648] HybridClock initialized: now 1786344239327702 us; error 40 us; skew 500 ppm
I20260810 06:43:59.328104 17141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:59 dist-test-slave-zj8q krb5kdc[17125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344239, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:59.333642 17141 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:43:59.335050 17141 webserver.cc:460] Webserver started at http://127.10.242.62:34537/ using document root <none> and password file <none>
I20260810 06:43:59.335760 17141 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:59.335875 17141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:59.336187 17141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:59.339186 17141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "211a70fd1cba4d588bd62160203d75b7"
format_stamp: "Formatted at 2026-08-10 06:43:59 on dist-test-slave-zj8q"
server_key: "9ddc7f384a3d4730fdc31b85b96938a9"
server_key_iv: "a35866c6355ec2fd10b5ea832ceea5d7"
server_key_version: "encryptionkey@0"
I20260810 06:43:59.339974 17141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "211a70fd1cba4d588bd62160203d75b7"
format_stamp: "Formatted at 2026-08-10 06:43:59 on dist-test-slave-zj8q"
server_key: "9ddc7f384a3d4730fdc31b85b96938a9"
server_key_iv: "a35866c6355ec2fd10b5ea832ceea5d7"
server_key_version: "encryptionkey@0"
I20260810 06:43:59.345053 17141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:43:59.348412 17168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:59.349684 17141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:43:59.349840 17141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "211a70fd1cba4d588bd62160203d75b7"
format_stamp: "Formatted at 2026-08-10 06:43:59 on dist-test-slave-zj8q"
server_key: "9ddc7f384a3d4730fdc31b85b96938a9"
server_key_iv: "a35866c6355ec2fd10b5ea832ceea5d7"
server_key_version: "encryptionkey@0"
I20260810 06:43:59.350018 17141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:59.391428 17141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:59.395278 17141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:59.395542 17141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:59.412631 17141 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:38713
I20260810 06:43:59.412652 17243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:38713 every 8 connection(s)
I20260810 06:43:59.413910 17141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:43:59.417485 17244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:43:59.419806 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 17141
I20260810 06:43:59.420017 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:43:59.420609 11208 external_mini_cluster.cc:1428] Setting key b7f6551260176d1ad7e931af93431283
I20260810 06:43:59.428915 17244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7: Bootstrap starting.
I20260810 06:43:59.431969 17244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7: Neither blocks nor log segments found. Creating new log.
I20260810 06:43:59.432909 17244 log.cc:824] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:43:59 dist-test-slave-zj8q krb5kdc[17125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:43:59.435536 17244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7: No bootstrap required, opened a new log
I20260810 06:43:59.441987 17248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:43:59.422500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.10.242.62:38713)
0810 06:43:59.423287 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0810 06:43:59.423299 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:43:59.423340 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:43:59.424996 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:43:59.425018 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:43:59.425403 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:43:59.425737 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:43:59.427529 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:59.428936 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:59.430239 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:43:59.430899 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:43:59.431072 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:43:59.434924 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:43:59.434932 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:43:59.434946 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:43:59.434983 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:43:59.437988 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:59.438704 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:59.438711 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:59.438717 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:59.438891 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:43:59.439271 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:43:59.439275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:43:59.439277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:43:59.439820 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:43:59.440107 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 06:43:59.440595 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 06:43:59.440987 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":369,"threads_started":1}
I20260810 06:43:59.443306 17244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a70fd1cba4d588bd62160203d75b7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38713 } }
I20260810 06:43:59.443641 17244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:43:59.443733 17244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 211a70fd1cba4d588bd62160203d75b7, State: Initialized, Role: FOLLOWER
I20260810 06:43:59.444281 17244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a70fd1cba4d588bd62160203d75b7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38713 } }
I20260810 06:43:59.444499 17244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:43:59.444646 17244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:43:59.444806 17244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:43:59.446892 17244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a70fd1cba4d588bd62160203d75b7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38713 } }
I20260810 06:43:59.447355 17244 leader_election.cc:302] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 211a70fd1cba4d588bd62160203d75b7; no voters:
I20260810 06:43:59.447943 17244 leader_election.cc:288] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:43:59.448064 17254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:43:59.448794 17254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [term 1 LEADER]: Becoming Leader. State: Replica: 211a70fd1cba4d588bd62160203d75b7, State: Running, Role: LEADER
I20260810 06:43:59.449311 17254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a70fd1cba4d588bd62160203d75b7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38713 } }
I20260810 06:43:59.449906 17244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:43:59.452075 17255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "211a70fd1cba4d588bd62160203d75b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a70fd1cba4d588bd62160203d75b7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38713 } } }
I20260810 06:43:59.452497 17255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:59.452059 17257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 211a70fd1cba4d588bd62160203d75b7. Latest consensus state: current_term: 1 leader_uuid: "211a70fd1cba4d588bd62160203d75b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a70fd1cba4d588bd62160203d75b7" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38713 } } }
I20260810 06:43:59.452767 17257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:43:59.455037 17261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:43:59.460369 17261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:43:59.469146 17261 catalog_manager.cc:1347] Generated new cluster ID: 2bf5b6eb63f0465a8a0be459210f6f08
I20260810 06:43:59.469257 17261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:43:59.500677 17261 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:43:59.502648 17261 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:43:59.514285 17261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 211a70fd1cba4d588bd62160203d75b7: Generated new TSK 0
I20260810 06:43:59.515321 17261 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:43:59.579941 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:38713
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:38203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:43:59.902899 17283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:43:59.903195 17283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:43:59.903340 17283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:43:59.910437 17283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:43:59.910552 17283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:43:59.910709 17283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:43:59.920312 17283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:38203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:38713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:43:59.921546 17283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:43:59.922889 17283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:43:59.935798 17290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:59.935902 17283 server_base.cc:1034] running on GCE node
W20260810 06:43:59.935804 17292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:43:59.935837 17295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:43:59.936977 17283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:43:59.937873 17283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:43:59.939065 17283 hybrid_clock.cc:648] HybridClock initialized: now 1786344239938989 us; error 62 us; skew 500 ppm
I20260810 06:43:59.939383 17283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:43:59 dist-test-slave-zj8q krb5kdc[17125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344239, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:43:59.942472 17283 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:43:59.943825 17283 webserver.cc:460] Webserver started at http://127.10.242.1:40705/ using document root <none> and password file <none>
I20260810 06:43:59.944494 17283 fs_manager.cc:359] Metadata directory not provided
I20260810 06:43:59.944612 17283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:43:59.944916 17283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:43:59.947852 17283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "5ddd1df7fd3746fa923691a64f69eb71"
format_stamp: "Formatted at 2026-08-10 06:43:59 on dist-test-slave-zj8q"
server_key: "bac9954df4263483720df948e7b31647"
server_key_iv: "e8357ff932a50328d4c6a25d63afcbef"
server_key_version: "encryptionkey@0"
I20260810 06:43:59.948603 17283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "5ddd1df7fd3746fa923691a64f69eb71"
format_stamp: "Formatted at 2026-08-10 06:43:59 on dist-test-slave-zj8q"
server_key: "bac9954df4263483720df948e7b31647"
server_key_iv: "e8357ff932a50328d4c6a25d63afcbef"
server_key_version: "encryptionkey@0"
I20260810 06:43:59.953790 17283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 06:43:59.957305 17306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:59.958619 17283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 06:43:59.958768 17283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "5ddd1df7fd3746fa923691a64f69eb71"
format_stamp: "Formatted at 2026-08-10 06:43:59 on dist-test-slave-zj8q"
server_key: "bac9954df4263483720df948e7b31647"
server_key_iv: "e8357ff932a50328d4c6a25d63afcbef"
server_key_version: "encryptionkey@0"
I20260810 06:43:59.958946 17283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:43:59.979290 17283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:43:59.983345 17283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:43:59.983608 17283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:43:59.984537 17283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:43:59.985850 17283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:43:59.985951 17283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:43:59.986068 17283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:43:59.986119 17283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:00.019374 17283 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:34899
I20260810 06:44:00.019400 17505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:34899 every 8 connection(s)
I20260810 06:44:00.020977 17283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:00.031363 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 17283
I20260810 06:44:00.031612 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:44:00.032537 11208 external_mini_cluster.cc:1428] Setting key 90e3bf67de0c1ea95827d362cd993c6d
Aug 10 06:44:00 dist-test-slave-zj8q krb5kdc[17125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344239, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:00.041433 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:00.023621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:48623 (local address 127.10.242.62:38713)
0810 06:44:00.024053 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:00.024058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:00.024951 (+ 893us) server_negotiation.cc:407] Connection header received
0810 06:44:00.026864 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:00.026870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:00.026954 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:00.027050 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:00.028564 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:00.029458 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:00.031090 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:00.031529 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:00.031692 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:00.035501 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:00.035507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:00.035511 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:00.035549 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:00.038589 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:00.039264 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:00.039268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:00.039271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:00.039420 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:00.039803 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:00.039807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:00.039810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:00.040113 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:00.040251 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:00.040900 (+ 649us) server_negotiation.cc:299] Negotiation successful
0810 06:44:00.041099 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":216,"threads_started":1}
I20260810 06:44:00.042323 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:00.024077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:38713 (local address 127.10.242.1:48623)
0810 06:44:00.024788 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:00.024837 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:00.026590 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:00.027311 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:00.027319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:00.027825 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:00.028394 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:00.028413 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:00.029622 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:00.029628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:00.030941 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:00.030959 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:00.031843 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:00.031851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:00.032194 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:00.032891 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:00.032929 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:00.035285 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:00.038710 (+ 3425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:00.038717 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:00.038730 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:00.039144 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:00.039512 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:00.039515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:00.039517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:00.039700 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:00.040215 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:00.040220 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:00.040521 (+ 301us) client_negotiation.cc:769] Sending connection context
0810 06:44:00.040764 (+ 243us) client_negotiation.cc:240] Negotiation successful
0810 06:44:00.041147 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":257,"threads_started":1}
I20260810 06:44:00.043802 17507 heartbeater.cc:347] Connected to a master server at 127.10.242.62:38713
I20260810 06:44:00.044085 17507 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:00.044896 17507 heartbeater.cc:511] Master 127.10.242.62:38713 requested a full tablet report, sending...
I20260810 06:44:00.047055 17192 ts_manager.cc:194] Registered new tserver with Master: 5ddd1df7fd3746fa923691a64f69eb71 (127.10.242.1:34899)
I20260810 06:44:00.048435 17192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:48623
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:44:00.082439 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:38713
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:38203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:00.409143 17514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:00.409415 17514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:00.409547 17514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:00.416741 17514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:00.416844 17514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:00.416987 17514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:00.426623 17514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:38203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:38713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:00.427772 17514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:00.429070 17514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:00.441985 17520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:00.442077 17525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:00.442006 17523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:00.442662 17514 server_base.cc:1034] running on GCE node
I20260810 06:44:00.443261 17514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:00.444159 17514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:00.445386 17514 hybrid_clock.cc:648] HybridClock initialized: now 1786344240445332 us; error 42 us; skew 500 ppm
I20260810 06:44:00.445710 17514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:00 dist-test-slave-zj8q krb5kdc[17125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344240, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:00.448812 17514 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:44:00.450296 17514 webserver.cc:460] Webserver started at http://127.10.242.2:36755/ using document root <none> and password file <none>
I20260810 06:44:00.451040 17514 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:00.451152 17514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:00.451462 17514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:00.454321 17514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "aa7e2b08376e48d38c740635b63f9afb"
format_stamp: "Formatted at 2026-08-10 06:44:00 on dist-test-slave-zj8q"
server_key: "ab11b80bacd53e8cbd9d49c84147100f"
server_key_iv: "a4682b8fffd5ad09f903ff41c6f286fc"
server_key_version: "encryptionkey@0"
I20260810 06:44:00.455093 17514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "aa7e2b08376e48d38c740635b63f9afb"
format_stamp: "Formatted at 2026-08-10 06:44:00 on dist-test-slave-zj8q"
server_key: "ab11b80bacd53e8cbd9d49c84147100f"
server_key_iv: "a4682b8fffd5ad09f903ff41c6f286fc"
server_key_version: "encryptionkey@0"
I20260810 06:44:00.459968 17514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:44:00.463415 17536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:00.464677 17514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:44:00.464829 17514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "aa7e2b08376e48d38c740635b63f9afb"
format_stamp: "Formatted at 2026-08-10 06:44:00 on dist-test-slave-zj8q"
server_key: "ab11b80bacd53e8cbd9d49c84147100f"
server_key_iv: "a4682b8fffd5ad09f903ff41c6f286fc"
server_key_version: "encryptionkey@0"
I20260810 06:44:00.465008 17514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:00.500573 17514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:00.504428 17514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:00.504688 17514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:00.505646 17514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:00.506956 17514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:00.507038 17514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:00.507159 17514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:00.507218 17514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:00.536999 17514 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:34117
I20260810 06:44:00.537068 17701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:34117 every 8 connection(s)
I20260810 06:44:00.538297 17514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:00.542542 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 17514
I20260810 06:44:00.542753 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:44:00.543285 11208 external_mini_cluster.cc:1428] Setting key 813b922186ff14a697b763e26b6d3a25
Aug 10 06:44:00 dist-test-slave-zj8q krb5kdc[17125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344240, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:00.559273 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:00.540628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:36817 (local address 127.10.242.62:38713)
0810 06:44:00.540856 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:00.540862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:00.541987 (+ 1125us) server_negotiation.cc:407] Connection header received
0810 06:44:00.544359 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:00.544365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:00.544449 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:00.544554 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:00.546381 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:00.547066 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:00.549040 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:00.549460 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:00.549553 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:00.553185 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:00.553191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:00.553195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:00.553241 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:00.556159 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:00.556904 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:00.556909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:00.556912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:00.557084 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:00.557484 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:00.557489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:00.557491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:00.557824 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:00.557969 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:00.558721 (+ 752us) server_negotiation.cc:299] Negotiation successful
0810 06:44:00.558969 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 06:44:00.559957 17704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:00.541146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:38713 (local address 127.10.242.2:36817)
0810 06:44:00.541820 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:00.541869 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:00.544058 (+ 2189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:00.544744 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:00.544752 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:00.545373 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:00.546192 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:00.546236 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:00.547209 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:00.547215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:00.548924 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:00.548938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:00.549688 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:00.549695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:00.549953 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:00.550658 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:00.550687 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:00.552974 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:00.556279 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:00.556286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:00.556297 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:00.556774 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:00.557195 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:00.557197 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:00.557200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:00.557370 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:00.557926 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:00.557932 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:00.558338 (+ 406us) client_negotiation.cc:769] Sending connection context
0810 06:44:00.558551 (+ 213us) client_negotiation.cc:240] Negotiation successful
0810 06:44:00.558919 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":257,"threads_started":1}
I20260810 06:44:00.561242 17702 heartbeater.cc:347] Connected to a master server at 127.10.242.62:38713
I20260810 06:44:00.561586 17702 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:00.562486 17702 heartbeater.cc:511] Master 127.10.242.62:38713 requested a full tablet report, sending...
I20260810 06:44:00.564078 17192 ts_manager.cc:194] Registered new tserver with Master: aa7e2b08376e48d38c740635b63f9afb (127.10.242.2:34117)
I20260810 06:44:00.564874 17192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:36817
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:44:00.594483 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:38713
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:38203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:00.911698 17711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:00.911978 17711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:00.912114 17711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:00.919134 17711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:00.919240 17711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:00.919389 17711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:00.928591 17711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:38203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:38713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:00.929816 17711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:00.931142 17711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:00.943497 17718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:00.943504 17717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:00.943750 17722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:00.944360 17711 server_base.cc:1034] running on GCE node
I20260810 06:44:00.944942 17711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:00.945815 17711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:00.947016 17711 hybrid_clock.cc:648] HybridClock initialized: now 1786344240946969 us; error 43 us; skew 500 ppm
I20260810 06:44:00.947326 17711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:00 dist-test-slave-zj8q krb5kdc[17125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344240, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:00.950390 17711 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:44:00.951776 17711 webserver.cc:460] Webserver started at http://127.10.242.3:44789/ using document root <none> and password file <none>
I20260810 06:44:00.952428 17711 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:00.952548 17711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:00.952839 17711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:00.955751 17711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "2b67c811ca2140c3b1f1dea8fbf37399"
format_stamp: "Formatted at 2026-08-10 06:44:00 on dist-test-slave-zj8q"
server_key: "adba2c3185625bc59ae8ee92dd6e0e52"
server_key_iv: "dd849b6f8aa17a3ddd3b9d147e434dc9"
server_key_version: "encryptionkey@0"
I20260810 06:44:00.956504 17711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "2b67c811ca2140c3b1f1dea8fbf37399"
format_stamp: "Formatted at 2026-08-10 06:44:00 on dist-test-slave-zj8q"
server_key: "adba2c3185625bc59ae8ee92dd6e0e52"
server_key_iv: "dd849b6f8aa17a3ddd3b9d147e434dc9"
server_key_version: "encryptionkey@0"
I20260810 06:44:00.961328 17711 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 06:44:00.964867 17730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:00.966131 17711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:44:00.966346 17711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "2b67c811ca2140c3b1f1dea8fbf37399"
format_stamp: "Formatted at 2026-08-10 06:44:00 on dist-test-slave-zj8q"
server_key: "adba2c3185625bc59ae8ee92dd6e0e52"
server_key_iv: "dd849b6f8aa17a3ddd3b9d147e434dc9"
server_key_version: "encryptionkey@0"
I20260810 06:44:00.966526 17711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:00.983273 17711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:00.986937 17711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:00.987213 17711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:00.988092 17711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:00.989389 17711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:00.989470 17711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:00.989589 17711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:00.989650 17711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:01.019249 17711 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:43371
I20260810 06:44:01.019268 17915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:43371 every 8 connection(s)
I20260810 06:44:01.020531 17711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:01.022574 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 17711
I20260810 06:44:01.022778 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:44:01.023335 11208 external_mini_cluster.cc:1428] Setting key 8790061baf4871efb0c2c4b8f7442478
Aug 10 06:44:01 dist-test-slave-zj8q krb5kdc[17125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344240, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:01.040066 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:01.023076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:59431 (local address 127.10.242.62:38713)
0810 06:44:01.023286 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:01.023292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:01.024629 (+ 1337us) server_negotiation.cc:407] Connection header received
0810 06:44:01.026836 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:01.026840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:01.026903 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:01.026974 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:01.028337 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.029100 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.030629 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.031120 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.031215 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:01.034531 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:01.034536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:01.034539 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:01.034572 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:01.037117 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.037783 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.037787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.037790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.037920 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.038389 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.038393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.038395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.038669 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:01.038812 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:01.039498 (+ 686us) server_negotiation.cc:299] Negotiation successful
0810 06:44:01.039732 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 06:44:01.040970 17918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:01.023628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:38713 (local address 127.10.242.3:59431)
0810 06:44:01.024458 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:01.024508 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:01.026571 (+ 2063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:01.027163 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:01.027171 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:01.027651 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:01.028208 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:01.028227 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.029218 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.029223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:01.030519 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:01.030532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.031314 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.031319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:01.031509 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:01.032079 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:01.032105 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:01.034335 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:01.037240 (+ 2905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:01.037249 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:01.037264 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:01.037659 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:01.038025 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:01.038030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:01.038034 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:01.038277 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:01.038799 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:01.038806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:01.039126 (+ 320us) client_negotiation.cc:769] Sending connection context
0810 06:44:01.039378 (+ 252us) client_negotiation.cc:240] Negotiation successful
0810 06:44:01.039867 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":307,"threads_started":1}
I20260810 06:44:01.042095 17916 heartbeater.cc:347] Connected to a master server at 127.10.242.62:38713
I20260810 06:44:01.042418 17916 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:01.043115 17916 heartbeater.cc:511] Master 127.10.242.62:38713 requested a full tablet report, sending...
I20260810 06:44:01.044432 17192 ts_manager.cc:194] Registered new tserver with Master: 2b67c811ca2140c3b1f1dea8fbf37399 (127.10.242.3:43371)
I20260810 06:44:01.045210 17192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:59431
I20260810 06:44:01.050796 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:44:01.051764 17507 heartbeater.cc:503] Master 127.10.242.62:38713 was elected leader, sending a full tablet report...
Aug 10 06:44:01 dist-test-slave-zj8q krb5kdc[17125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:44:01 dist-test-slave-zj8q krb5kdc[17125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:44:01.103941 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:01.088973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.10.242.1:34899)
0810 06:44:01.089671 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:01.089681 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:01.089715 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:44:01.089783 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:01.089798 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:01.090041 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:01.090216 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:01.091681 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.092739 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.093960 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.094579 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.094695 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:01.098505 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:01.098516 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:01.098532 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:01.098597 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:01.101183 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.101862 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.101870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.101873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.102002 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.102403 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.102406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.102408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.102786 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:01.103019 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:01.103230 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 06:44:01.103502 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":272,"threads_started":1}
Aug 10 06:44:01 dist-test-slave-zj8q krb5kdc[17125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:01.121119 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:01.108470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.10.242.62:38713)
0810 06:44:01.108717 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:01.108723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:01.108830 (+ 107us) server_negotiation.cc:407] Connection header received
0810 06:44:01.108962 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:01.108966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:01.109029 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:01.109104 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:01.110240 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.110963 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.112168 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.112571 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.112661 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:01.116353 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:01.116358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:01.116362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:01.116397 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:01.118736 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.119354 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.119357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.119360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.119486 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.119837 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.119840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.119842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.120149 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:01.120332 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:01.120609 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 06:44:01.120828 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260810 06:44:01.121662 17192 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40780
I20260810 06:44:01.122738 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 17283
I20260810 06:44:01.135317 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:01.123542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.10.242.1:34899)
0810 06:44:01.123729 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:01.123736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:01.123971 (+ 235us) server_negotiation.cc:407] Connection header received
0810 06:44:01.124151 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:01.124156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:01.124251 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:01.124342 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:01.125535 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.126281 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.127525 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.127936 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.128047 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:01.130300 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:01.130306 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:01.130311 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:01.130347 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:01.132844 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.133450 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.133455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.133458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.133585 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.133953 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.133956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.133959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.134258 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:01.134494 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:01.134776 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 06:44:01.134994 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:44:01.135905 17500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:01.217767 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 17283
I20260810 06:44:01.235110 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 17514
Aug 10 06:44:01 dist-test-slave-zj8q krb5kdc[17125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:44:01.249966 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:01.236024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.10.242.2:34117)
0810 06:44:01.236551 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:01.236558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:01.236583 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:44:01.236674 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:01.236681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:01.236893 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:01.237017 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:01.238222 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.239251 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.240433 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.240971 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.241066 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:01.245022 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:01.245033 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:01.245048 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:01.245095 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:01.247690 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.248265 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.248271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.248273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.248392 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.248675 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.248678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.248680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.249002 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:01.249218 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:01.249453 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 06:44:01.249657 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":264,"threads_started":1}
I20260810 06:44:01.251466 17697 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:01.332899 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 17514
I20260810 06:44:01.360792 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 17711
Aug 10 06:44:01 dist-test-slave-zj8q krb5kdc[17125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:44:01.376929 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:01.361728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.10.242.3:43371)
0810 06:44:01.362377 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:01.362385 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:01.362417 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:44:01.362483 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:01.362489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:01.362715 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:01.362866 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:01.364185 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.365251 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.366575 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:01.367138 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:01.367253 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:01.371427 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:01.371438 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:01.371454 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:01.371489 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:01.374182 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.374853 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.374861 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.374863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.374992 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:01.375345 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:01.375348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:01.375351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:01.375723 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:01.376025 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:01.376333 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 06:44:01.376595 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":309,"threads_started":1}
I20260810 06:44:01.378237 17904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:01.458940 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 17711
I20260810 06:44:01.477008 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 17141
I20260810 06:44:01.478215 17239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:01.580188 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 17141
2026-08-10T06:44:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4043 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:44:01 dist-test-slave-zj8q krb5kdc[17948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:44:01 dist-test-slave-zj8q krb5kdc[17948](info): set up 2 sockets
Aug 10 06:44:01 dist-test-slave-zj8q krb5kdc[17948](info): commencing operation
krb5kdc: starting...
W20260810 06:44:02.804018 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:44:02 dist-test-slave-zj8q krb5kdc[17948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:44:03.003072 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40015
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:40015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:03.325042 17969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:03.325321 17969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:03.325466 17969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:03.332598 17969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:03.332703 17969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:03.332775 17969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:03.332852 17969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:03.332916 17969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:03.342980 17969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:40015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:03.344434 17969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:03.345891 17969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:03.357148 17985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:03.357162 17977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:03.357206 17980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:03.357975 17969 server_base.cc:1034] running on GCE node
I20260810 06:44:03.358754 17969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:03.360034 17969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:03.361263 17969 hybrid_clock.cc:648] HybridClock initialized: now 1786344243361213 us; error 32 us; skew 500 ppm
I20260810 06:44:03.361673 17969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:03 dist-test-slave-zj8q krb5kdc[17948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344243, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:03.364816 17969 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:44:03.365868 17969 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:03.365981 17969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:03.366358 17969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:03.369221 17969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "3f2e07b8130f43a3b6663b62ebf9821a"
format_stamp: "Formatted at 2026-08-10 06:44:03 on dist-test-slave-zj8q"
server_key: "a7cbc177a4adbcf41599c4a142a8694b"
server_key_iv: "48984b810ea210acf6f81434a80969da"
server_key_version: "encryptionkey@0"
I20260810 06:44:03.369997 17969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "3f2e07b8130f43a3b6663b62ebf9821a"
format_stamp: "Formatted at 2026-08-10 06:44:03 on dist-test-slave-zj8q"
server_key: "a7cbc177a4adbcf41599c4a142a8694b"
server_key_iv: "48984b810ea210acf6f81434a80969da"
server_key_version: "encryptionkey@0"
I20260810 06:44:03.374866 17969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 06:44:03.378191 17996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:03.379437 17969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:44:03.379593 17969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "3f2e07b8130f43a3b6663b62ebf9821a"
format_stamp: "Formatted at 2026-08-10 06:44:03 on dist-test-slave-zj8q"
server_key: "a7cbc177a4adbcf41599c4a142a8694b"
server_key_iv: "48984b810ea210acf6f81434a80969da"
server_key_version: "encryptionkey@0"
I20260810 06:44:03.379793 17969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:03.408674 17969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:03.412420 17969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:03.412662 17969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:03.429440 17969 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:40015
I20260810 06:44:03.429415 18075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:40015 every 8 connection(s)
I20260810 06:44:03.430367 17969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:03.431449 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 17969
I20260810 06:44:03.431656 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:03.432273 11208 external_mini_cluster.cc:1428] Setting key 8de1eb5d8e8796de3fb3ee8b68824361
I20260810 06:44:03.434405 18076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:44:03 dist-test-slave-zj8q krb5kdc[17948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:03.447350 18076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a: Bootstrap starting.
I20260810 06:44:03.451344 18076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:03.452567 18076 log.cc:824] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:03.455369 18076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a: No bootstrap required, opened a new log
I20260810 06:44:03.455618 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:03.434188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.10.242.62:40015)
0810 06:44:03.435031 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:03.435046 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:03.435095 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:44:03.437243 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:03.437271 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:03.437757 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:03.438204 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:03.439688 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:03.441688 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:03.442958 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:03.443946 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:03.444156 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:03.447872 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:03.447883 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:03.447901 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:03.447943 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:03.451610 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:03.452259 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:03.452268 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:03.452276 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:03.452489 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:03.452858 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:03.452863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:03.452866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:03.453507 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:03.453807 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:03.454287 (+ 480us) server_negotiation.cc:299] Negotiation successful
0810 06:44:03.454677 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":354,"threads_started":1}
I20260810 06:44:03.463469 18076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2e07b8130f43a3b6663b62ebf9821a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40015 } }
I20260810 06:44:03.463830 18076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:03.463934 18076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2e07b8130f43a3b6663b62ebf9821a, State: Initialized, Role: FOLLOWER
I20260810 06:44:03.464447 18076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2e07b8130f43a3b6663b62ebf9821a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40015 } }
I20260810 06:44:03.464665 18076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:03.464792 18076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:03.464942 18076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:03.467402 18076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2e07b8130f43a3b6663b62ebf9821a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40015 } }
I20260810 06:44:03.467826 18076 leader_election.cc:302] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f2e07b8130f43a3b6663b62ebf9821a; no voters:
I20260810 06:44:03.468365 18076 leader_election.cc:288] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:03.468495 18086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:03.469434 18086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [term 1 LEADER]: Becoming Leader. State: Replica: 3f2e07b8130f43a3b6663b62ebf9821a, State: Running, Role: LEADER
I20260810 06:44:03.469996 18086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2e07b8130f43a3b6663b62ebf9821a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40015 } }
I20260810 06:44:03.470497 18076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:03.472874 18088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f2e07b8130f43a3b6663b62ebf9821a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2e07b8130f43a3b6663b62ebf9821a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40015 } } }
I20260810 06:44:03.473291 18088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:03.473500 18089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f2e07b8130f43a3b6663b62ebf9821a. Latest consensus state: current_term: 1 leader_uuid: "3f2e07b8130f43a3b6663b62ebf9821a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f2e07b8130f43a3b6663b62ebf9821a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40015 } } }
I20260810 06:44:03.473802 18089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:03.479135 18097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:03.483243 18097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:03.491063 18097 catalog_manager.cc:1347] Generated new cluster ID: 437464dfe39e44de9043524333a206bd
I20260810 06:44:03.491184 18097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:03.503209 18097 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:03.505012 18097 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:03.518070 18097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a: Generated new TSK 0
I20260810 06:44:03.519098 18097 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:44:03.570111 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40015
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:03.897512 18114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:03.897766 18114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:03.897902 18114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:03.904969 18114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:03.905069 18114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:03.905227 18114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:44:03.914654 18114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:03.915874 18114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:03.917217 18114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:03.929694 18125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:03.929580 18133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:03.929631 18126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:03.930467 18114 server_base.cc:1034] running on GCE node
I20260810 06:44:03.931051 18114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:03.931929 18114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:03.933139 18114 hybrid_clock.cc:648] HybridClock initialized: now 1786344243933089 us; error 32 us; skew 500 ppm
I20260810 06:44:03.933441 18114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:03 dist-test-slave-zj8q krb5kdc[17948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344243, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:03.936549 18114 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:44:03.937585 18114 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:03.937683 18114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:03.937949 18114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:03.940829 18114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "5c2c594fae2e43c3ac291077c572e04a"
format_stamp: "Formatted at 2026-08-10 06:44:03 on dist-test-slave-zj8q"
server_key: "823062f92731f3f9a7bd3c0b20ee9639"
server_key_iv: "6fdbf02b321f5593cca5db7986047253"
server_key_version: "encryptionkey@0"
I20260810 06:44:03.941591 18114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "5c2c594fae2e43c3ac291077c572e04a"
format_stamp: "Formatted at 2026-08-10 06:44:03 on dist-test-slave-zj8q"
server_key: "823062f92731f3f9a7bd3c0b20ee9639"
server_key_iv: "6fdbf02b321f5593cca5db7986047253"
server_key_version: "encryptionkey@0"
I20260810 06:44:03.946447 18114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 06:44:03.949662 18139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:03.950901 18114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:03.951059 18114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "5c2c594fae2e43c3ac291077c572e04a"
format_stamp: "Formatted at 2026-08-10 06:44:03 on dist-test-slave-zj8q"
server_key: "823062f92731f3f9a7bd3c0b20ee9639"
server_key_iv: "6fdbf02b321f5593cca5db7986047253"
server_key_version: "encryptionkey@0"
I20260810 06:44:03.951252 18114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:03.964740 18114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:03.968289 18114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:03.968524 18114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:03.969408 18114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:03.970736 18114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:03.970829 18114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:03.970932 18114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:03.970999 18114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:04.000032 18114 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:36627
I20260810 06:44:04.000092 18333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:36627 every 8 connection(s)
I20260810 06:44:04.000893 18114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:04.002100 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 18114
I20260810 06:44:04.002357 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:44:04.003207 11208 external_mini_cluster.cc:1428] Setting key a81a48d30d1bd9d38d9716210ac4bc13
Aug 10 06:44:04 dist-test-slave-zj8q krb5kdc[17948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344243, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:04.022663 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:04.003674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:33301 (local address 127.10.242.62:40015)
0810 06:44:04.004135 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:04.004141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:04.006229 (+ 2088us) server_negotiation.cc:407] Connection header received
0810 06:44:04.007115 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:04.007122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:04.007202 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:04.007296 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:04.008793 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.009497 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.011082 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.011492 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.011602 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:04.015508 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:04.015515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:04.015520 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:04.015585 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:04.019479 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:04.020272 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:04.020278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:04.020282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:04.020429 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:04.020905 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:04.020910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:04.020914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:04.021204 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:04.021348 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:04.022036 (+ 688us) server_negotiation.cc:299] Negotiation successful
0810 06:44:04.022304 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":218,"threads_started":1}
I20260810 06:44:04.023597 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:04.004229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40015 (local address 127.10.242.1:33301)
0810 06:44:04.005033 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:04.005085 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:04.006851 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:04.007538 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:04.007546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:04.008060 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:04.008622 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:04.008639 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.009640 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.009644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:04.010939 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:04.010953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.011715 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.011723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:04.011934 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:04.012532 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:04.012592 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:04.015225 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:04.019625 (+ 4400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:04.019634 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:04.019668 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:04.020121 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:04.020576 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:04.020581 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:04.020584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:04.020786 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:04.021323 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:04.021330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:04.021665 (+ 335us) client_negotiation.cc:769] Sending connection context
0810 06:44:04.021917 (+ 252us) client_negotiation.cc:240] Negotiation successful
0810 06:44:04.022457 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":343,"threads_started":1}
I20260810 06:44:04.025240 18334 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40015
I20260810 06:44:04.025599 18334 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:04.026489 18334 heartbeater.cc:511] Master 127.10.242.62:40015 requested a full tablet report, sending...
I20260810 06:44:04.029043 18023 ts_manager.cc:194] Registered new tserver with Master: 5c2c594fae2e43c3ac291077c572e04a (127.10.242.1:36627)
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
I20260810 06:44:04.030759 18023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:33301
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:44:04.056702 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40015
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:04.379665 18344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:04.379914 18344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:04.380018 18344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:04.387009 18344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:04.387104 18344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:04.387218 18344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:04.396216 18344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:04.397321 18344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:04.398590 18344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:04.410694 18350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:04.410851 18355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:04.411028 18349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:04.411362 18344 server_base.cc:1034] running on GCE node
I20260810 06:44:04.411949 18344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:04.412792 18344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:04.413993 18344 hybrid_clock.cc:648] HybridClock initialized: now 1786344244413952 us; error 30 us; skew 500 ppm
I20260810 06:44:04.414325 18344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:04 dist-test-slave-zj8q krb5kdc[17948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344244, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:04.419996 18344 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:44:04.421077 18344 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:04.421181 18344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:04.421484 18344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:04.424463 18344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "4b698b1fa7714797a3cc785c9d8f026f"
format_stamp: "Formatted at 2026-08-10 06:44:04 on dist-test-slave-zj8q"
server_key: "6c1a8fea84ac4082f61c4ac5d390b237"
server_key_iv: "468b3cd9c5a7d9318f65ab25e20bbcc3"
server_key_version: "encryptionkey@0"
I20260810 06:44:04.425256 18344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "4b698b1fa7714797a3cc785c9d8f026f"
format_stamp: "Formatted at 2026-08-10 06:44:04 on dist-test-slave-zj8q"
server_key: "6c1a8fea84ac4082f61c4ac5d390b237"
server_key_iv: "468b3cd9c5a7d9318f65ab25e20bbcc3"
server_key_version: "encryptionkey@0"
I20260810 06:44:04.430110 18344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:44:04.433418 18367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:04.434734 18344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:04.434888 18344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "4b698b1fa7714797a3cc785c9d8f026f"
format_stamp: "Formatted at 2026-08-10 06:44:04 on dist-test-slave-zj8q"
server_key: "6c1a8fea84ac4082f61c4ac5d390b237"
server_key_iv: "468b3cd9c5a7d9318f65ab25e20bbcc3"
server_key_version: "encryptionkey@0"
I20260810 06:44:04.435070 18344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:04.451257 18344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:04.454730 18344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:04.454973 18344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:04.455821 18344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:04.457056 18344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:04.457149 18344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:04.457250 18344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:04.457322 18344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:04.487210 18344 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:36887
I20260810 06:44:04.487272 18567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:36887 every 8 connection(s)
I20260810 06:44:04.487974 18344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:04.494715 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 18344
I20260810 06:44:04.494902 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:44:04.495514 11208 external_mini_cluster.cc:1428] Setting key 4630a5c0ae866aa8dc3660eff9ba981d
Aug 10 06:44:04 dist-test-slave-zj8q krb5kdc[17948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344244, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:04.507114 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:04.490191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:38677 (local address 127.10.242.62:40015)
0810 06:44:04.490387 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:04.490394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:04.491483 (+ 1089us) server_negotiation.cc:407] Connection header received
0810 06:44:04.493233 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:04.493237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:04.493310 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:04.493399 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:04.494893 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.495959 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.497546 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.497972 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.498068 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:04.501739 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:04.501744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:04.501748 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:04.501780 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:04.504184 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:04.504874 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:04.504879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:04.504881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:04.505019 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:04.505428 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:04.505432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:04.505434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:04.505708 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:04.505839 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:04.506550 (+ 711us) server_negotiation.cc:299] Negotiation successful
0810 06:44:04.506787 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:44:04.508176 18572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:04.490639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40015 (local address 127.10.242.2:38677)
0810 06:44:04.491323 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:04.491366 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:04.492979 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:04.493586 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:04.493594 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:04.494126 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:04.494704 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:04.494722 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.496163 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.496168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:04.497405 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:04.497421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.498229 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.498237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:04.498476 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:04.499133 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:04.499167 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:04.501532 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:04.504316 (+ 2784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:04.504325 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:04.504341 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:04.504753 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:04.505136 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:04.505141 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:04.505145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:04.505317 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:04.505831 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:04.505838 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:04.506183 (+ 345us) client_negotiation.cc:769] Sending connection context
0810 06:44:04.506427 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 06:44:04.506924 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":267,"threads_started":1}
I20260810 06:44:04.509471 18568 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40015
I20260810 06:44:04.509745 18568 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:04.510416 18568 heartbeater.cc:511] Master 127.10.242.62:40015 requested a full tablet report, sending...
I20260810 06:44:04.511798 18023 ts_manager.cc:194] Registered new tserver with Master: 4b698b1fa7714797a3cc785c9d8f026f (127.10.242.2:36887)
I20260810 06:44:04.512573 18023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:38677
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:44:04.542804 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40015
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:04.858749 18575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:04.859028 18575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:04.859167 18575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:04.866108 18575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:04.866250 18575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:04.866400 18575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:04.875872 18575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:04.877095 18575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:04.878402 18575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:04.890411 18585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:04.890460 18587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:04.890481 18590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:04.891544 18575 server_base.cc:1034] running on GCE node
I20260810 06:44:04.892055 18575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:04.892913 18575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:04.894099 18575 hybrid_clock.cc:648] HybridClock initialized: now 1786344244894055 us; error 38 us; skew 500 ppm
I20260810 06:44:04.894411 18575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:04 dist-test-slave-zj8q krb5kdc[17948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344244, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:04.897332 18575 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:44:04.898531 18575 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:04.898638 18575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:04.898940 18575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:04.901678 18575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "3fa104e907e6474095bfb827948c782f"
format_stamp: "Formatted at 2026-08-10 06:44:04 on dist-test-slave-zj8q"
server_key: "3bc4693310f3e214619ab49eb2122d48"
server_key_iv: "a0a98d655905e5432907fd18b9b7dda0"
server_key_version: "encryptionkey@0"
I20260810 06:44:04.902472 18575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "3fa104e907e6474095bfb827948c782f"
format_stamp: "Formatted at 2026-08-10 06:44:04 on dist-test-slave-zj8q"
server_key: "3bc4693310f3e214619ab49eb2122d48"
server_key_iv: "a0a98d655905e5432907fd18b9b7dda0"
server_key_version: "encryptionkey@0"
I20260810 06:44:04.907097 18575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:44:04.910239 18599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:04.911418 18575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:44:04.911557 18575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "3fa104e907e6474095bfb827948c782f"
format_stamp: "Formatted at 2026-08-10 06:44:04 on dist-test-slave-zj8q"
server_key: "3bc4693310f3e214619ab49eb2122d48"
server_key_iv: "a0a98d655905e5432907fd18b9b7dda0"
server_key_version: "encryptionkey@0"
I20260810 06:44:04.911746 18575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:04.943727 18575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:04.947784 18575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:04.948010 18575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:04.948925 18575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:04.950256 18575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:04.950344 18575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:04.950428 18575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:04.950479 18575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:04.980466 18575 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:33883
I20260810 06:44:04.980505 18792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:33883 every 8 connection(s)
I20260810 06:44:04.981356 18575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:04.990689 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 18575
I20260810 06:44:04.990907 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:44:04.991479 11208 external_mini_cluster.cc:1428] Setting key 11ee43193ad9c83e4bb09eb498380762
Aug 10 06:44:04 dist-test-slave-zj8q krb5kdc[17948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344244, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:05.001654 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:04.983891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:35283 (local address 127.10.242.62:40015)
0810 06:44:04.984083 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:04.984088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:04.985267 (+ 1179us) server_negotiation.cc:407] Connection header received
0810 06:44:04.987196 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:04.987203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:04.987286 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:04.987376 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:04.988951 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.989716 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.991898 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.992397 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.992528 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:04.996217 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:04.996222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:04.996225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:04.996257 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:04.998645 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:04.999327 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:04.999331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:04.999334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:04.999483 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:04.999928 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:04.999932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:04.999936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:05.000281 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:05.000431 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.001095 (+ 664us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.001322 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:44:05.002583 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:04.984388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40015 (local address 127.10.242.3:35283)
0810 06:44:04.985101 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:04.985140 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:04.986891 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:04.987629 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:04.987637 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:04.988175 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:04.988771 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:04.988790 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.989866 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.989870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:04.991746 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:04.991763 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:04.992680 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:04.992689 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:04.992913 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:04.993686 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:04.993718 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:04.996031 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:04.998770 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:04.998776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:04.998788 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:04.999220 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:04.999622 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:04.999625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:04.999629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:04.999814 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:05.000418 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:05.000424 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:05.000735 (+ 311us) client_negotiation.cc:769] Sending connection context
0810 06:44:05.000960 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 06:44:05.001322 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":277,"threads_started":1}
I20260810 06:44:05.003991 18797 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40015
I20260810 06:44:05.004307 18797 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:05.004980 18797 heartbeater.cc:511] Master 127.10.242.62:40015 requested a full tablet report, sending...
I20260810 06:44:05.006596 18023 ts_manager.cc:194] Registered new tserver with Master: 3fa104e907e6474095bfb827948c782f (127.10.242.3:33883)
I20260810 06:44:05.007102 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:44:05.007498 18023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:35283
I20260810 06:44:05.024338 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.011945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49248 (local address 127.10.242.62:40015)
0810 06:44:05.012153 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.012160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.012644 (+ 484us) server_negotiation.cc:407] Connection header received
0810 06:44:05.012818 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.012824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.012890 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.012969 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:05.014457 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.015190 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.016529 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.016968 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.017081 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.019318 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:05.019323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:05.019327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:05.019359 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:05.021809 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:05.022458 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:05.022464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:05.022468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:05.022596 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:05.022961 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:05.022965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:05.022967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:05.023280 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:05.023497 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.023836 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.024043 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 06:44:05.028306 18023 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 06:44:05.034065 18334 heartbeater.cc:503] Master 127.10.242.62:40015 was elected leader, sending a full tablet report...
W20260810 06:44:05.035297 18023 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:44:05.081977 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.047167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:33883 (local address 127.0.0.1:36260)
0810 06:44:05.072696 (+ 25529us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:05.073464 (+ 768us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:05.073695 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:05.074737 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:05.074742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:05.074765 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:05.075246 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.075259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.077194 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.077198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.079272 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.079285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.080743 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.080750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.080932 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.080986 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 06:44:05.081325 (+ 339us) client_negotiation.cc:240] Negotiation successful
0810 06:44:05.081507 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25191,"mutex_wait_us":55,"thread_start_us":25524,"threads_started":1}
I20260810 06:44:05.082600 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.048203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53688 (local address 127.10.242.1:36627)
0810 06:44:05.071320 (+ 23117us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.071331 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.074352 (+ 3021us) server_negotiation.cc:407] Connection header received
0810 06:44:05.074631 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.074639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.074856 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.075007 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:05.075783 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.076823 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.080085 (+ 3262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.081495 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.081614 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.081845 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.082097 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.082289 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22876,"thread_start_us":23128,"threads_started":1}
I20260810 06:44:05.083019 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.047687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36260 (local address 127.10.242.3:33883)
0810 06:44:05.072092 (+ 24405us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.072102 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.073577 (+ 1475us) server_negotiation.cc:407] Connection header received
0810 06:44:05.073893 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.073903 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.074249 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.074425 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:05.075441 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.077012 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.079417 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.080544 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.080672 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.081090 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.082517 (+ 1427us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.082677 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24205,"thread_start_us":24048,"threads_started":1}
I20260810 06:44:05.082661 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.047572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:36627 (local address 127.0.0.1:53688)
0810 06:44:05.074237 (+ 26665us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:05.074254 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:05.074454 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:05.075286 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:05.075290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:05.075315 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:05.075606 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.075613 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.076959 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.076963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.079911 (+ 2948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.079927 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.081722 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.081729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.081929 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.081953 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:44:05.082034 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:44:05.082174 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26502,"mutex_wait_us":26,"thread_start_us":25700,"threads_started":1}
I20260810 06:44:05.082875 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.047238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:36887 (local address 127.0.0.1:54428)
0810 06:44:05.072696 (+ 25458us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:05.072765 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:05.072996 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:05.074487 (+ 1491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:05.074494 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:05.074521 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:05.074980 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.074994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.077153 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.077158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.079911 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.079927 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.081849 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.081860 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.082026 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.082057 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:44:05.082138 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:44:05.082273 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25220,"mutex_wait_us":287,"thread_start_us":26689,"threads_started":1}
I20260810 06:44:05.083766 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.047604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54428 (local address 127.10.242.2:36887)
0810 06:44:05.073831 (+ 26227us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.073839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.073871 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:44:05.073970 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.073978 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.074253 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.075222 (+ 969us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:05.075417 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.077026 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.080114 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.081254 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.081370 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.083193 (+ 1823us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.083275 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.083431 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26089,"thread_start_us":25978,"threads_started":1}
I20260810 06:44:05.087248 18451 tablet_service.cc:1467] Processing CreateTablet for tablet 0bcb8fe297e84b7fa921a7c742277414 (DEFAULT_TABLE table=test-table [id=5098c6cd9faf471b854c86be69715839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:05.087277 18690 tablet_service.cc:1467] Processing CreateTablet for tablet 0bcb8fe297e84b7fa921a7c742277414 (DEFAULT_TABLE table=test-table [id=5098c6cd9faf471b854c86be69715839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:05.087277 18227 tablet_service.cc:1467] Processing CreateTablet for tablet 0bcb8fe297e84b7fa921a7c742277414 (DEFAULT_TABLE table=test-table [id=5098c6cd9faf471b854c86be69715839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:05.088438 18451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bcb8fe297e84b7fa921a7c742277414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:05.088603 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bcb8fe297e84b7fa921a7c742277414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:05.088611 18227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bcb8fe297e84b7fa921a7c742277414. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:05.099378 18827 tablet_bootstrap.cc:492] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: Bootstrap starting.
I20260810 06:44:05.102864 18830 tablet_bootstrap.cc:492] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: Bootstrap starting.
I20260810 06:44:05.103454 18827 tablet_bootstrap.cc:654] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:05.104810 18827 log.cc:824] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:05.106261 18830 tablet_bootstrap.cc:654] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:05.106249 18828 tablet_bootstrap.cc:492] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: Bootstrap starting.
I20260810 06:44:05.107511 18830 log.cc:824] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:05.107851 18827 tablet_bootstrap.cc:492] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: No bootstrap required, opened a new log
I20260810 06:44:05.108078 18827 ts_tablet_manager.cc:1397] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:44:05.109908 18828 tablet_bootstrap.cc:654] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:05.111137 18828 log.cc:824] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:05.111265 18830 tablet_bootstrap.cc:492] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: No bootstrap required, opened a new log
I20260810 06:44:05.111567 18830 ts_tablet_manager.cc:1397] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260810 06:44:05.114202 18828 tablet_bootstrap.cc:492] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: No bootstrap required, opened a new log
I20260810 06:44:05.114459 18828 ts_tablet_manager.cc:1397] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 06:44:05.116439 18827 raft_consensus.cc:348] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } }
I20260810 06:44:05.116954 18827 raft_consensus.cc:374] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:05.117110 18827 raft_consensus.cc:729] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b698b1fa7714797a3cc785c9d8f026f, State: Initialized, Role: FOLLOWER
I20260810 06:44:05.117766 18827 consensus_queue.cc:260] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } }
I20260810 06:44:05.118829 18568 heartbeater.cc:503] Master 127.10.242.62:40015 was elected leader, sending a full tablet report...
I20260810 06:44:05.119549 18827 ts_tablet_manager.cc:1428] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260810 06:44:05.121459 18830 raft_consensus.cc:348] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } }
I20260810 06:44:05.121903 18830 raft_consensus.cc:374] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:05.122009 18830 raft_consensus.cc:729] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa104e907e6474095bfb827948c782f, State: Initialized, Role: FOLLOWER
I20260810 06:44:05.122641 18830 consensus_queue.cc:260] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } }
I20260810 06:44:05.123785 18797 heartbeater.cc:503] Master 127.10.242.62:40015 was elected leader, sending a full tablet report...
I20260810 06:44:05.124415 18830 ts_tablet_manager.cc:1428] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 06:44:05.125923 18828 raft_consensus.cc:348] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } }
I20260810 06:44:05.126439 18828 raft_consensus.cc:374] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:05.126549 18828 raft_consensus.cc:729] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c2c594fae2e43c3ac291077c572e04a, State: Initialized, Role: FOLLOWER
I20260810 06:44:05.127064 18828 consensus_queue.cc:260] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } }
I20260810 06:44:05.128971 18828 ts_tablet_manager.cc:1428] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260810 06:44:05.177291 18835 raft_consensus.cc:482] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:05.177596 18835 raft_consensus.cc:504] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } }
I20260810 06:44:05.179412 18835 leader_election.cc:288] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fa104e907e6474095bfb827948c782f (127.10.242.3:33883), 5c2c594fae2e43c3ac291077c572e04a (127.10.242.1:36627)
I20260810 06:44:05.186048 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.180083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:55821 (local address 127.10.242.1:36627)
0810 06:44:05.180285 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.180291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.180850 (+ 559us) server_negotiation.cc:407] Connection header received
0810 06:44:05.180996 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.181000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.181058 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.181129 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:05.181789 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.182592 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.184434 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.185157 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.185315 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.185394 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.185687 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.185810 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:44:05.186178 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.179829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:40199 (local address 127.10.242.3:33883)
0810 06:44:05.180036 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.180043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.180735 (+ 692us) server_negotiation.cc:407] Connection header received
0810 06:44:05.180921 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.180925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.180985 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.181059 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:05.181712 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.182505 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.184470 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.185254 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.185404 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.185477 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.185792 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.185917 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:44:05.186209 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.179882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:36627 (local address 127.10.242.2:55821)
0810 06:44:05.180731 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:05.180746 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:05.180869 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:05.181157 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:05.181162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:05.181189 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:05.181625 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.181636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.182726 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.182731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.184287 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.184302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.185375 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.185381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.185533 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.185557 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:44:05.185644 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:44:05.185772 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":20,"thread_start_us":560,"threads_started":1}
I20260810 06:44:05.186452 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.179757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:33883 (local address 127.10.242.2:40199)
0810 06:44:05.180623 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:05.180643 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:05.180791 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:05.181092 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:05.181096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:05.181121 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:05.181544 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.181553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.182643 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.182648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.184312 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:05.184325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.185475 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.185481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:05.185635 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.185663 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 06:44:05.185743 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:44:05.185858 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":20,"spinlock_wait_cycles":19328,"thread_start_us":462,"threads_started":1}
I20260810 06:44:05.187203 18259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcb8fe297e84b7fa921a7c742277414" candidate_uuid: "4b698b1fa7714797a3cc785c9d8f026f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c2c594fae2e43c3ac291077c572e04a" is_pre_election: true
I20260810 06:44:05.187394 18725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcb8fe297e84b7fa921a7c742277414" candidate_uuid: "4b698b1fa7714797a3cc785c9d8f026f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa104e907e6474095bfb827948c782f" is_pre_election: true
I20260810 06:44:05.187721 18259 raft_consensus.cc:2454] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b698b1fa7714797a3cc785c9d8f026f in term 0.
I20260810 06:44:05.187867 18725 raft_consensus.cc:2454] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b698b1fa7714797a3cc785c9d8f026f in term 0.
I20260810 06:44:05.188378 18372 leader_election.cc:302] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b698b1fa7714797a3cc785c9d8f026f, 5c2c594fae2e43c3ac291077c572e04a; no voters:
I20260810 06:44:05.188761 18835 raft_consensus.cc:2781] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:44:05.188895 18835 raft_consensus.cc:482] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:05.188982 18835 raft_consensus.cc:3037] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:05.191156 18835 raft_consensus.cc:504] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } }
I20260810 06:44:05.191696 18835 leader_election.cc:288] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [CANDIDATE]: Term 1 election: Requested vote from peers 3fa104e907e6474095bfb827948c782f (127.10.242.3:33883), 5c2c594fae2e43c3ac291077c572e04a (127.10.242.1:36627)
I20260810 06:44:05.192078 18725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcb8fe297e84b7fa921a7c742277414" candidate_uuid: "4b698b1fa7714797a3cc785c9d8f026f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa104e907e6474095bfb827948c782f"
I20260810 06:44:05.192103 18259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcb8fe297e84b7fa921a7c742277414" candidate_uuid: "4b698b1fa7714797a3cc785c9d8f026f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c2c594fae2e43c3ac291077c572e04a"
I20260810 06:44:05.192252 18725 raft_consensus.cc:3037] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:05.192271 18259 raft_consensus.cc:3037] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:05.194424 18725 raft_consensus.cc:2454] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b698b1fa7714797a3cc785c9d8f026f in term 1.
I20260810 06:44:05.194473 18259 raft_consensus.cc:2454] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b698b1fa7714797a3cc785c9d8f026f in term 1.
I20260810 06:44:05.194788 18369 leader_election.cc:302] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fa104e907e6474095bfb827948c782f, 4b698b1fa7714797a3cc785c9d8f026f; no voters:
I20260810 06:44:05.195091 18835 raft_consensus.cc:2781] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:05.195511 18835 raft_consensus.cc:686] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 1 LEADER]: Becoming Leader. State: Replica: 4b698b1fa7714797a3cc785c9d8f026f, State: Running, Role: LEADER
I20260810 06:44:05.195955 18835 consensus_queue.cc:237] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } }
I20260810 06:44:05.200657 18023 catalog_manager.cc:5495] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b698b1fa7714797a3cc785c9d8f026f (127.10.242.2). New cstate: current_term: 1 leader_uuid: "4b698b1fa7714797a3cc785c9d8f026f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b698b1fa7714797a3cc785c9d8f026f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 } health_report { overall_health: UNKNOWN } } }
W20260810 06:44:05.232923 18798 tablet.cc:2369] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:44:05.239671 18570 tablet.cc:2369] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:44:05.242946 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.237393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.10.242.2:36887)
0810 06:44:05.237580 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.237584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.237664 (+ 80us) server_negotiation.cc:407] Connection header received
0810 06:44:05.237866 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.237870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.237994 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.238193 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:05.238506 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.239302 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.241015 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.241429 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.241525 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.241943 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:05.242508 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:05.242616 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.242684 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.242749 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:44:05.250829 18259 raft_consensus.cc:1261] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [term 1 FOLLOWER]: Refusing update from remote peer 4b698b1fa7714797a3cc785c9d8f026f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:05.250869 18725 raft_consensus.cc:1261] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [term 1 FOLLOWER]: Refusing update from remote peer 4b698b1fa7714797a3cc785c9d8f026f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:05.251744 18835 consensus_queue.cc:1035] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c2c594fae2e43c3ac291077c572e04a" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:05.252019 18840 consensus_queue.cc:1035] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fa104e907e6474095bfb827948c782f" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 33883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:44:05.253108 18335 tablet.cc:2369] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:44:05.262714 18844 mvcc.cc:205] Tried to move back new op lower bound from 7316866028538159104 to 7316866028326240256. Current Snapshot: MvccSnapshot[applied={T|T < 7316866028538159104}]
I20260810 06:44:05.265162 18847 mvcc.cc:205] Tried to move back new op lower bound from 7316866028538159104 to 7316866028326240256. Current Snapshot: MvccSnapshot[applied={T|T < 7316866028538159104}]
I20260810 06:44:05.265470 18846 mvcc.cc:205] Tried to move back new op lower bound from 7316866028538159104 to 7316866028326240256. Current Snapshot: MvccSnapshot[applied={T|T < 7316866028538159104}]
I20260810 06:44:05.278645 18022 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:05.278898 18022 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:05.282248 18022 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f2e07b8130f43a3b6663b62ebf9821a: Sending DeleteTablet for 3 replicas of tablet 0bcb8fe297e84b7fa921a7c742277414
I20260810 06:44:05.283324 18690 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bcb8fe297e84b7fa921a7c742277414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:05 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:36260
I20260810 06:44:05.283545 18451 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bcb8fe297e84b7fa921a7c742277414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:05 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:54428
I20260810 06:44:05.284220 18227 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bcb8fe297e84b7fa921a7c742277414 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:05 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:53688
I20260810 06:44:05.284848 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 18114
I20260810 06:44:05.298770 18858 tablet_replica.cc:330] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: stopping tablet replica
I20260810 06:44:05.299305 18858 raft_consensus.cc:2229] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:44:05.300590 18859 tablet_replica.cc:330] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: stopping tablet replica
I20260810 06:44:05.301170 18859 raft_consensus.cc:2229] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:05.301797 18859 raft_consensus.cc:2258] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:05.302745 18857 tablet_replica.cc:330] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: stopping tablet replica
I20260810 06:44:05.303074 18857 raft_consensus.cc:2229] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:05.303141 18858 raft_consensus.cc:2258] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:05.303488 18857 raft_consensus.cc:2258] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:05.305378 18859 ts_tablet_manager.cc:1905] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:05.305647 18857 ts_tablet_manager.cc:1905] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:05.305810 18858 ts_tablet_manager.cc:1905] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:44:05 dist-test-slave-zj8q krb5kdc[17948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:44:05.312381 18857 ts_tablet_manager.cc:1918] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:05.312500 18859 ts_tablet_manager.cc:1918] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:05.312597 18857 log.cc:1196] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/0bcb8fe297e84b7fa921a7c742277414
I20260810 06:44:05.312703 18859 log.cc:1196] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/0bcb8fe297e84b7fa921a7c742277414
I20260810 06:44:05.313179 18857 ts_tablet_manager.cc:1939] T 0bcb8fe297e84b7fa921a7c742277414 P 3fa104e907e6474095bfb827948c782f: Deleting consensus metadata
I20260810 06:44:05.313232 18859 ts_tablet_manager.cc:1939] T 0bcb8fe297e84b7fa921a7c742277414 P 5c2c594fae2e43c3ac291077c572e04a: Deleting consensus metadata
I20260810 06:44:05.315224 17997 catalog_manager.cc:4841] TS 3fa104e907e6474095bfb827948c782f (127.10.242.3:33883): tablet 0bcb8fe297e84b7fa921a7c742277414 (table test-table [id=5098c6cd9faf471b854c86be69715839]) successfully deleted
I20260810 06:44:05.315194 18001 catalog_manager.cc:4841] TS 5c2c594fae2e43c3ac291077c572e04a (127.10.242.1:36627): tablet 0bcb8fe297e84b7fa921a7c742277414 (table test-table [id=5098c6cd9faf471b854c86be69715839]) successfully deleted
I20260810 06:44:05.317756 18858 ts_tablet_manager.cc:1918] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:05.317997 18858 log.cc:1196] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/0bcb8fe297e84b7fa921a7c742277414
I20260810 06:44:05.318013 18825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.286203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53702 (local address 127.10.242.1:36627)
0810 06:44:05.286353 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.286360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.295974 (+ 9614us) server_negotiation.cc:407] Connection header received
0810 06:44:05.296428 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.296434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.296526 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.296614 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:05.299608 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.300837 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.304631 (+ 3794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.305264 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.305370 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.310630 (+ 5260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:05.310656 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:05.310678 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:05.310726 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:05.314693 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:05.315634 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:05.315654 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:05.315658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:05.315820 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:05.316302 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:05.316308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:05.316311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:05.316794 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:05.317118 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.317428 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.317648 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:44:05.318538 18858 ts_tablet_manager.cc:1939] T 0bcb8fe297e84b7fa921a7c742277414 P 4b698b1fa7714797a3cc785c9d8f026f: Deleting consensus metadata
I20260810 06:44:05.318787 18328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:05.320066 17998 catalog_manager.cc:4841] TS 4b698b1fa7714797a3cc785c9d8f026f (127.10.242.2:36887): tablet 0bcb8fe297e84b7fa921a7c742277414 (table test-table [id=5098c6cd9faf471b854c86be69715839]) successfully deleted
I20260810 06:44:05.427412 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 18114
I20260810 06:44:05.459206 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 18344
Aug 10 06:44:05 dist-test-slave-zj8q krb5kdc[17948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:44:05.474861 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.460023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.10.242.2:36887)
0810 06:44:05.460267 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.460273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.460292 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:44:05.460442 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.460446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.460520 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.460603 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:05.461873 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.462748 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.464012 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.464453 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.464568 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.468833 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:05.468848 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:05.468865 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:05.468900 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:05.471930 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:05.472596 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:05.472607 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:05.472610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:05.472792 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:05.473146 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:05.473152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:05.473156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:05.473666 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:05.474011 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.474227 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.474452 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260810 06:44:05.475761 18563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:05.589040 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 18344
I20260810 06:44:05.615872 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 18575
Aug 10 06:44:05 dist-test-slave-zj8q krb5kdc[17948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:44:05.630414 18823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:05.616596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.10.242.3:33883)
0810 06:44:05.616803 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:05.616807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:05.616827 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:44:05.616986 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:05.616990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:05.617057 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:05.617137 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:05.618429 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.619157 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.620361 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:05.620739 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:05.620826 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:05.624905 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:05.624925 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:05.624948 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:05.625012 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:05.627654 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:05.628327 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:05.628336 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:05.628338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:05.628477 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:05.628848 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:05.628853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:05.628855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:05.629210 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:05.629498 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:05.629847 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 06:44:05.630065 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 06:44:05.631032 18786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:05.718528 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 18575
I20260810 06:44:05.742815 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 17969
I20260810 06:44:05.743924 18070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:05.852291 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 17969
2026-08-10T06:44:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4265 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T06:44:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:05Z Disabled control of system clock
I20260810 06:44:05.926915 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40023
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:39909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:40023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:44:06.253019 18875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:06.253307 18875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:06.253404 18875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:06.260555 18875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:06.260658 18875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:06.260707 18875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:06.260762 18875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:06.260805 18875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:06.269760 18875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:40023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:06.270999 18875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:06.272746 18875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:06.284026 18887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:06.284137 18884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:06.284344 18883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:06.284965 18875 server_base.cc:1034] running on GCE node
I20260810 06:44:06.285730 18875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:06.287081 18875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:06.288280 18875 hybrid_clock.cc:648] HybridClock initialized: now 1786344246288243 us; error 27 us; skew 500 ppm
I20260810 06:44:06.288606 18875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:06.290758 18875 webserver.cc:460] Webserver started at http://127.10.242.62:42063/ using document root <none> and password file <none>
I20260810 06:44:06.291424 18875 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:06.291523 18875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:06.291851 18875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:06.294703 18875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "ce8f2951da8b4b23943e4aad4bbc084e"
format_stamp: "Formatted at 2026-08-10 06:44:06 on dist-test-slave-zj8q"
server_key: "895b24108ad7d228609b1f5c2ded957e"
server_key_iv: "1df071331074da8080cda235380833a4"
server_key_version: "encryptionkey@0"
I20260810 06:44:06.295471 18875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "ce8f2951da8b4b23943e4aad4bbc084e"
format_stamp: "Formatted at 2026-08-10 06:44:06 on dist-test-slave-zj8q"
server_key: "895b24108ad7d228609b1f5c2ded957e"
server_key_iv: "1df071331074da8080cda235380833a4"
server_key_version: "encryptionkey@0"
I20260810 06:44:06.300352 18875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260810 06:44:06.304953 18898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:06.306345 18875 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:44:06.306492 18875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "ce8f2951da8b4b23943e4aad4bbc084e"
format_stamp: "Formatted at 2026-08-10 06:44:06 on dist-test-slave-zj8q"
server_key: "895b24108ad7d228609b1f5c2ded957e"
server_key_iv: "1df071331074da8080cda235380833a4"
server_key_version: "encryptionkey@0"
I20260810 06:44:06.306706 18875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:06.312052 18875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:06.312767 18875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:06.313040 18875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:06.329767 18875 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:40023
I20260810 06:44:06.329810 18980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:40023 every 8 connection(s)
I20260810 06:44:06.331007 18875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:06.334488 18981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:06.339015 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 18875
I20260810 06:44:06.339185 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:06.339829 11208 external_mini_cluster.cc:1428] Setting key a3710e3aa0fdf8024ab1357607c7bf54
I20260810 06:44:06.344749 18981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e: Bootstrap starting.
I20260810 06:44:06.347609 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:06.341709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.10.242.62:40023)
0810 06:44:06.342727 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:06.342737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:06.342776 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:44:06.344339 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:06.344361 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:06.344731 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:06.345042 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:06.346112 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:06.346118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:06.346147 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:06.346191 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:06.346262 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:06.346280 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:06.346449 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:06.346544 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 06:44:06.346751 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":368,"threads_started":1}
I20260810 06:44:06.348126 18981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:06.349102 18981 log.cc:824] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:06.351706 18981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e: No bootstrap required, opened a new log
I20260810 06:44:06.358726 18981 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8f2951da8b4b23943e4aad4bbc084e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40023 } }
I20260810 06:44:06.359030 18981 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:06.359099 18981 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce8f2951da8b4b23943e4aad4bbc084e, State: Initialized, Role: FOLLOWER
I20260810 06:44:06.359583 18981 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8f2951da8b4b23943e4aad4bbc084e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40023 } }
I20260810 06:44:06.359793 18981 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:06.359886 18981 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:06.360001 18981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:06.361958 18981 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8f2951da8b4b23943e4aad4bbc084e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40023 } }
I20260810 06:44:06.362406 18981 leader_election.cc:302] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce8f2951da8b4b23943e4aad4bbc084e; no voters:
I20260810 06:44:06.362926 18981 leader_election.cc:288] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:06.363032 18991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:06.364117 18991 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [term 1 LEADER]: Becoming Leader. State: Replica: ce8f2951da8b4b23943e4aad4bbc084e, State: Running, Role: LEADER
I20260810 06:44:06.364624 18991 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8f2951da8b4b23943e4aad4bbc084e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40023 } }
I20260810 06:44:06.364934 18981 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:06.367340 18993 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce8f2951da8b4b23943e4aad4bbc084e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8f2951da8b4b23943e4aad4bbc084e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40023 } } }
I20260810 06:44:06.367316 18996 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce8f2951da8b4b23943e4aad4bbc084e. Latest consensus state: current_term: 1 leader_uuid: "ce8f2951da8b4b23943e4aad4bbc084e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce8f2951da8b4b23943e4aad4bbc084e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40023 } } }
I20260810 06:44:06.367775 18993 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:06.367777 18996 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:06.368646 19005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:06.374193 19005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:06.381659 19005 catalog_manager.cc:1347] Generated new cluster ID: a005b8ce54b14c149ed0d2f43c2312c1
I20260810 06:44:06.381758 19005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:06.401116 19005 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:06.401475 19005 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:06.414112 19005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e: Generated new TSK 0
I20260810 06:44:06.414937 19005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:44:06.429519 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40023
--builtin_ntp_servers=127.10.242.20:39909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:44:06.753999 19021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:06.754323 19021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:06.754464 19021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:06.761547 19021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:06.761653 19021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:06.761807 19021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:44:06.771178 19021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:06.772480 19021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:06.774358 19021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:06.787052 19031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:06.787082 19030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:06.787004 19033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:06.787756 19021 server_base.cc:1034] running on GCE node
I20260810 06:44:06.788338 19021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:06.789227 19021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:06.790419 19021 hybrid_clock.cc:648] HybridClock initialized: now 1786344246790376 us; error 35 us; skew 500 ppm
I20260810 06:44:06.790725 19021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:06.792886 19021 webserver.cc:460] Webserver started at http://127.10.242.1:38061/ using document root <none> and password file <none>
I20260810 06:44:06.793567 19021 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:06.793668 19021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:06.793989 19021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:06.796800 19021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "53d6b2f9b88a460b8f6b6a1517799857"
format_stamp: "Formatted at 2026-08-10 06:44:06 on dist-test-slave-zj8q"
server_key: "dba5ea22a942507e070b3af7f3740939"
server_key_iv: "b9ea8444bdda0bc8ee7350252e521157"
server_key_version: "encryptionkey@0"
I20260810 06:44:06.797580 19021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "53d6b2f9b88a460b8f6b6a1517799857"
format_stamp: "Formatted at 2026-08-10 06:44:06 on dist-test-slave-zj8q"
server_key: "dba5ea22a942507e070b3af7f3740939"
server_key_iv: "b9ea8444bdda0bc8ee7350252e521157"
server_key_version: "encryptionkey@0"
I20260810 06:44:06.802490 19021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:44:06.805691 19043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:06.806932 19021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:06.807082 19021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "53d6b2f9b88a460b8f6b6a1517799857"
format_stamp: "Formatted at 2026-08-10 06:44:06 on dist-test-slave-zj8q"
server_key: "dba5ea22a942507e070b3af7f3740939"
server_key_iv: "b9ea8444bdda0bc8ee7350252e521157"
server_key_version: "encryptionkey@0"
I20260810 06:44:06.807287 19021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:06.812726 19021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:06.813457 19021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:06.813699 19021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:06.814591 19021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:06.815898 19021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:06.815990 19021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:06.816090 19021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:06.816167 19021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:06.846277 19021 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:45999
I20260810 06:44:06.846298 19216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:45999 every 8 connection(s)
I20260810 06:44:06.847599 19021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:06.850945 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 19021
I20260810 06:44:06.851119 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:44:06.851753 11208 external_mini_cluster.cc:1428] Setting key f18fc00883687a542d2110ddd95e2313
I20260810 06:44:06.854642 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40023
--builtin_ntp_servers=127.10.242.20:39909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 06:44:06.856170 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:06.850048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:39089 (local address 127.10.242.62:40023)
0810 06:44:06.850536 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:06.850542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:06.852273 (+ 1731us) server_negotiation.cc:407] Connection header received
0810 06:44:06.853962 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:06.853967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:06.854031 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:06.854147 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:06.855446 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:06.855452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:06.855456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:06.855476 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:06.855529 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:06.855544 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:06.855679 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:06.855814 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:44:06.855877 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":227,"threads_started":1}
I20260810 06:44:06.856976 19221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:06.850661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40023 (local address 127.10.242.1:39089)
0810 06:44:06.851695 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:06.851747 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:06.853676 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:06.854387 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:06.854395 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:06.854486 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:06.855138 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:06.855140 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:06.855179 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:06.855269 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:06.855645 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:06.855651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:06.855657 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:44:06.855859 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 06:44:06.856053 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":316,"threads_started":1}
I20260810 06:44:06.858387 19217 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40023
I20260810 06:44:06.858698 19217 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:06.859390 19217 heartbeater.cc:511] Master 127.10.242.62:40023 requested a full tablet report, sending...
I20260810 06:44:06.861472 18923 ts_manager.cc:194] Registered new tserver with Master: 53d6b2f9b88a460b8f6b6a1517799857 (127.10.242.1:45999)
W20260810 06:44:07.179747 19222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:07.180012 19222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:07.180104 19222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:07.187208 19222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:07.187294 19222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:07.187405 19222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:07.196066 19222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:07.197268 19222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:07.199020 19222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:07.211519 19232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:07.211454 19233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:07.211547 19238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:07.212899 19222 server_base.cc:1034] running on GCE node
I20260810 06:44:07.213439 19222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:07.214329 19222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:07.215528 19222 hybrid_clock.cc:648] HybridClock initialized: now 1786344247215486 us; error 32 us; skew 500 ppm
I20260810 06:44:07.215838 19222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:07.218999 19222 webserver.cc:460] Webserver started at http://127.10.242.2:44863/ using document root <none> and password file <none>
I20260810 06:44:07.219813 19222 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:07.219905 19222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:07.220189 19222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:07.222947 19222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "d39ee393a2ea432eb35fa88fa16bd644"
format_stamp: "Formatted at 2026-08-10 06:44:07 on dist-test-slave-zj8q"
server_key: "3d4aea3e2f9855ece6e6b2eb7a272273"
server_key_iv: "2b12c125816674b15f18f6b13936bc68"
server_key_version: "encryptionkey@0"
I20260810 06:44:07.223667 19222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "d39ee393a2ea432eb35fa88fa16bd644"
format_stamp: "Formatted at 2026-08-10 06:44:07 on dist-test-slave-zj8q"
server_key: "3d4aea3e2f9855ece6e6b2eb7a272273"
server_key_iv: "2b12c125816674b15f18f6b13936bc68"
server_key_version: "encryptionkey@0"
I20260810 06:44:07.228447 19222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 06:44:07.231590 19249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:07.232785 19222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:44:07.232935 19222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "d39ee393a2ea432eb35fa88fa16bd644"
format_stamp: "Formatted at 2026-08-10 06:44:07 on dist-test-slave-zj8q"
server_key: "3d4aea3e2f9855ece6e6b2eb7a272273"
server_key_iv: "2b12c125816674b15f18f6b13936bc68"
server_key_version: "encryptionkey@0"
I20260810 06:44:07.233129 19222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:07.238461 19222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:07.239180 19222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:07.239414 19222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:07.240242 19222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:07.241568 19222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:07.241662 19222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:07.241765 19222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:07.241837 19222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:07.271399 19222 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:36171
I20260810 06:44:07.271462 19423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:36171 every 8 connection(s)
I20260810 06:44:07.272609 19222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:07.277236 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 19222
I20260810 06:44:07.277413 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:44:07.277949 11208 external_mini_cluster.cc:1428] Setting key 1760c01405b27fc6cccc98c1500d0859
I20260810 06:44:07.280635 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.274866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:60495 (local address 127.10.242.62:40023)
0810 06:44:07.275020 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.275024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.276332 (+ 1308us) server_negotiation.cc:407] Connection header received
0810 06:44:07.278429 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.278434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.278492 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.278594 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.279872 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.279877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.279880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.279897 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.279930 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.279941 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.280090 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.280257 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.280328 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:44:07.281038 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40023
--builtin_ntp_servers=127.10.242.20:39909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 06:44:07.281517 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.275318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40023 (local address 127.10.242.2:60495)
0810 06:44:07.276189 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:07.276223 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:07.278110 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:07.278788 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:07.278795 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:07.278879 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:07.279550 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:07.279553 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:07.279594 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:07.279681 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:07.280051 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:07.280063 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:07.280071 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:44:07.280337 (+ 266us) client_negotiation.cc:240] Negotiation successful
0810 06:44:07.280579 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":269,"threads_started":1}
I20260810 06:44:07.282774 19425 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40023
I20260810 06:44:07.283043 19425 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:07.283581 19425 heartbeater.cc:511] Master 127.10.242.62:40023 requested a full tablet report, sending...
I20260810 06:44:07.285030 18923 ts_manager.cc:194] Registered new tserver with Master: d39ee393a2ea432eb35fa88fa16bd644 (127.10.242.2:36171)
W20260810 06:44:07.604580 19429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:07.604848 19429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:07.604974 19429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:07.612072 19429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:07.612162 19429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:07.612304 19429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:07.621012 19429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:07.622303 19429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:07.624075 19429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:07.636354 19434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:07.636298 19438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:07.636328 19435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:07.637212 19429 server_base.cc:1034] running on GCE node
I20260810 06:44:07.637873 19429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:07.638813 19429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:07.640014 19429 hybrid_clock.cc:648] HybridClock initialized: now 1786344247639961 us; error 49 us; skew 500 ppm
I20260810 06:44:07.640319 19429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:07.642488 19429 webserver.cc:460] Webserver started at http://127.10.242.3:39021/ using document root <none> and password file <none>
I20260810 06:44:07.643148 19429 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:07.643249 19429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:07.643558 19429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:07.646324 19429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "270dd939658740e89a862ebbf6f04be5"
format_stamp: "Formatted at 2026-08-10 06:44:07 on dist-test-slave-zj8q"
server_key: "62da5d4fa8795472efedc89833243b18"
server_key_iv: "5d93ab3915ed61a0575cb57503c7cd88"
server_key_version: "encryptionkey@0"
I20260810 06:44:07.647099 19429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "270dd939658740e89a862ebbf6f04be5"
format_stamp: "Formatted at 2026-08-10 06:44:07 on dist-test-slave-zj8q"
server_key: "62da5d4fa8795472efedc89833243b18"
server_key_iv: "5d93ab3915ed61a0575cb57503c7cd88"
server_key_version: "encryptionkey@0"
I20260810 06:44:07.651960 19429 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:44:07.655169 19444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:07.656464 19429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:44:07.656661 19429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "270dd939658740e89a862ebbf6f04be5"
format_stamp: "Formatted at 2026-08-10 06:44:07 on dist-test-slave-zj8q"
server_key: "62da5d4fa8795472efedc89833243b18"
server_key_iv: "5d93ab3915ed61a0575cb57503c7cd88"
server_key_version: "encryptionkey@0"
I20260810 06:44:07.656859 19429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:07.662129 19429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:07.662969 19429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:07.663206 19429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:07.664063 19429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:07.665381 19429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:07.665477 19429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:07.665596 19429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:07.665669 19429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:07.694931 19429 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:46529
I20260810 06:44:07.694984 19624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:46529 every 8 connection(s)
I20260810 06:44:07.696225 19429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:07.701848 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 19429
I20260810 06:44:07.701993 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:44:07.702596 11208 external_mini_cluster.cc:1428] Setting key 48f0776582537e58c5c7e2b2190e1132
I20260810 06:44:07.704207 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.698690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:58619 (local address 127.10.242.62:40023)
0810 06:44:07.698874 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.698878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.700192 (+ 1314us) server_negotiation.cc:407] Connection header received
0810 06:44:07.701827 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.701833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.701897 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.701996 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.703486 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.703491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.703494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.703512 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.703552 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.703570 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.703701 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.703874 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.703941 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:44:07.705195 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.699172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40023 (local address 127.10.242.3:58619)
0810 06:44:07.700035 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:07.700076 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:07.701560 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:07.702353 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:07.702364 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:07.702479 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:07.703185 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:07.703188 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:07.703223 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:07.703311 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:07.703674 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:07.703682 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:07.703688 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:44:07.703945 (+ 257us) client_negotiation.cc:240] Negotiation successful
0810 06:44:07.704202 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":252,"threads_started":1}
I20260810 06:44:07.706418 19625 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40023
I20260810 06:44:07.706683 19625 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:07.707222 19625 heartbeater.cc:511] Master 127.10.242.62:40023 requested a full tablet report, sending...
I20260810 06:44:07.708446 18923 ts_manager.cc:194] Registered new tserver with Master: 270dd939658740e89a862ebbf6f04be5 (127.10.242.3:46529)
I20260810 06:44:07.717660 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:44:07.724176 19219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.721809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.10.242.62:40023)
0810 06:44:07.721953 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.721957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.722477 (+ 520us) server_negotiation.cc:407] Connection header received
0810 06:44:07.722638 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.722643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.722694 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.722772 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.723598 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.723604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.723609 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.723625 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.723655 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.723666 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.723770 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.723876 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.723933 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:44:07.727761 18923 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:44:07.734381 18923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:44:07.753877 19644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.745636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45999 (local address 127.0.0.1:40392)
0810 06:44:07.747089 (+ 1453us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:07.747136 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:07.747269 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:07.751744 (+ 4475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:07.751751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:07.751836 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:07.752551 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:07.752555 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:07.752585 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:07.752673 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:07.753118 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:07.753133 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:07.753141 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:44:07.753305 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 06:44:07.753406 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"thread_start_us":5508,"threads_started":1}
I20260810 06:44:07.755012 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.745986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:46529 (local address 127.0.0.1:44750)
0810 06:44:07.751431 (+ 5445us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:07.751452 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:07.751550 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:07.753039 (+ 1489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:07.753044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:07.753066 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:07.753711 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:07.753714 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:07.753730 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:07.753776 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:07.754373 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:07.754378 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:07.754383 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:44:07.754538 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 06:44:07.754589 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5306,"thread_start_us":1017,"threads_started":1}
I20260810 06:44:07.755020 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.745841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.10.242.1:45999)
0810 06:44:07.751146 (+ 5305us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.751154 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.751184 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:44:07.751257 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.751264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.751502 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.751736 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.752877 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.752886 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.752901 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.752928 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.752984 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.753005 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.754543 (+ 1538us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.754616 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.754725 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5151,"thread_start_us":5025,"threads_started":1}
I20260810 06:44:07.755684 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.746685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44750 (local address 127.10.242.3:46529)
0810 06:44:07.750733 (+ 4048us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.750743 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.751639 (+ 896us) server_negotiation.cc:407] Connection header received
0810 06:44:07.751713 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.751721 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.751922 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.752069 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.753973 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.753983 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.753998 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.754016 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.754068 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.754109 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.754314 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.755239 (+ 925us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.755330 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3844,"thread_start_us":3671,"threads_started":1}
I20260810 06:44:07.756929 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.752781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:36171 (local address 127.0.0.1:38438)
0810 06:44:07.754408 (+ 1627us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:07.754432 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:07.754531 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:07.755322 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:07.755326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:07.755347 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:07.755969 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:07.755972 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:07.755986 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:07.756026 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:07.756361 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:07.756366 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:07.756371 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:44:07.756500 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 06:44:07.756549 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1453,"thread_start_us":774,"threads_started":1}
I20260810 06:44:07.758509 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.751804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38438 (local address 127.10.242.2:36171)
0810 06:44:07.754086 (+ 2282us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.754094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.754658 (+ 564us) server_negotiation.cc:407] Connection header received
0810 06:44:07.754734 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.754741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.754947 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.755081 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.756169 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.756175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.756184 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.756207 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.756247 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.756264 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.756433 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.756503 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.756591 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2150,"thread_start_us":2038,"threads_started":1}
I20260810 06:44:07.759167 19524 tablet_service.cc:1467] Processing CreateTablet for tablet c8ace20e21ea48439dd011a886f00853 (DEFAULT_TABLE table=test-table [id=b5396bfd8bb94967afbdc018c64650a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:07.759174 19123 tablet_service.cc:1467] Processing CreateTablet for tablet c8ace20e21ea48439dd011a886f00853 (DEFAULT_TABLE table=test-table [id=b5396bfd8bb94967afbdc018c64650a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:07.760648 19524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ace20e21ea48439dd011a886f00853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:07.760641 19123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ace20e21ea48439dd011a886f00853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:07.761739 19326 tablet_service.cc:1467] Processing CreateTablet for tablet c8ace20e21ea48439dd011a886f00853 (DEFAULT_TABLE table=test-table [id=b5396bfd8bb94967afbdc018c64650a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:07.762835 19326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ace20e21ea48439dd011a886f00853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:07.773834 19654 tablet_bootstrap.cc:492] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: Bootstrap starting.
I20260810 06:44:07.777441 19654 tablet_bootstrap.cc:654] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:07.778452 19654 log.cc:824] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:07.780653 19656 tablet_bootstrap.cc:492] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857: Bootstrap starting.
I20260810 06:44:07.780977 19655 tablet_bootstrap.cc:492] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: Bootstrap starting.
I20260810 06:44:07.781551 19654 tablet_bootstrap.cc:492] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: No bootstrap required, opened a new log
I20260810 06:44:07.781793 19654 ts_tablet_manager.cc:1397] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 06:44:07.784204 19656 tablet_bootstrap.cc:654] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:07.784883 19655 tablet_bootstrap.cc:654] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:07.785414 19656 log.cc:824] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:07.786144 19655 log.cc:824] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:07.788559 19656 tablet_bootstrap.cc:492] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857: No bootstrap required, opened a new log
I20260810 06:44:07.788841 19656 ts_tablet_manager.cc:1397] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260810 06:44:07.789490 19654 raft_consensus.cc:348] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } }
I20260810 06:44:07.789845 19654 raft_consensus.cc:374] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:07.789944 19654 raft_consensus.cc:729] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d39ee393a2ea432eb35fa88fa16bd644, State: Initialized, Role: FOLLOWER
I20260810 06:44:07.790539 19654 consensus_queue.cc:260] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } }
I20260810 06:44:07.791143 19655 tablet_bootstrap.cc:492] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: No bootstrap required, opened a new log
I20260810 06:44:07.791507 19425 heartbeater.cc:503] Master 127.10.242.62:40023 was elected leader, sending a full tablet report...
I20260810 06:44:07.791987 19654 ts_tablet_manager.cc:1428] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260810 06:44:07.792232 19655 ts_tablet_manager.cc:1397] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260810 06:44:07.797168 19656 raft_consensus.cc:348] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } }
I20260810 06:44:07.797482 19656 raft_consensus.cc:374] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:07.797576 19656 raft_consensus.cc:729] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d6b2f9b88a460b8f6b6a1517799857, State: Initialized, Role: FOLLOWER
I20260810 06:44:07.798169 19656 consensus_queue.cc:260] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } }
I20260810 06:44:07.799161 19217 heartbeater.cc:503] Master 127.10.242.62:40023 was elected leader, sending a full tablet report...
I20260810 06:44:07.799574 19656 ts_tablet_manager.cc:1428] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260810 06:44:07.801123 19655 raft_consensus.cc:348] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } }
I20260810 06:44:07.801512 19655 raft_consensus.cc:374] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:07.801622 19655 raft_consensus.cc:729] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270dd939658740e89a862ebbf6f04be5, State: Initialized, Role: FOLLOWER
I20260810 06:44:07.802206 19655 consensus_queue.cc:260] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } }
I20260810 06:44:07.803282 19625 heartbeater.cc:503] Master 127.10.242.62:40023 was elected leader, sending a full tablet report...
I20260810 06:44:07.803756 19655 ts_tablet_manager.cc:1428] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260810 06:44:07.814590 19663 raft_consensus.cc:482] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:07.814847 19663 raft_consensus.cc:504] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } }
I20260810 06:44:07.816406 19663 leader_election.cc:288] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 270dd939658740e89a862ebbf6f04be5 (127.10.242.3:46529), d39ee393a2ea432eb35fa88fa16bd644 (127.10.242.2:36171)
I20260810 06:44:07.819540 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.816740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:51205 (local address 127.10.242.3:46529)
0810 06:44:07.816927 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.816933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.817682 (+ 749us) server_negotiation.cc:407] Connection header received
0810 06:44:07.817834 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.817840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.817892 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.817970 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.818948 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.818954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.818959 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.818978 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.819027 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.819043 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.819157 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.819240 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.819301 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:44:07.819656 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.816895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:54703 (local address 127.10.242.2:36171)
0810 06:44:07.817111 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.817118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.817801 (+ 683us) server_negotiation.cc:407] Connection header received
0810 06:44:07.817992 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.817996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.818042 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.818119 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.819056 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.819061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.819066 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.819083 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.819112 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.819123 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.819226 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.819339 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.819398 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 06:44:07.819906 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.816551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:46529 (local address 127.10.242.1:51205)
0810 06:44:07.817580 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:07.817596 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:07.817717 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:07.818011 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:07.818015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:07.818039 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:07.818736 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:07.818740 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:07.818758 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:07.818808 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:07.819137 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:07.819142 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:07.819147 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:44:07.819299 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 06:44:07.819438 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":586,"threads_started":1}
I20260810 06:44:07.820025 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.816815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:36171 (local address 127.10.242.1:54703)
0810 06:44:07.817706 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:07.817724 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:07.817877 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:07.818189 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:07.818215 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:07.818239 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:07.818874 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:07.818876 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:07.818889 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:07.818924 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:07.819221 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:07.819225 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:07.819230 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:44:07.819369 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 06:44:07.819414 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":513,"threads_started":1}
I20260810 06:44:07.820928 19357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ace20e21ea48439dd011a886f00853" candidate_uuid: "53d6b2f9b88a460b8f6b6a1517799857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d39ee393a2ea432eb35fa88fa16bd644" is_pre_election: true
I20260810 06:44:07.820883 19553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ace20e21ea48439dd011a886f00853" candidate_uuid: "53d6b2f9b88a460b8f6b6a1517799857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270dd939658740e89a862ebbf6f04be5" is_pre_election: true
I20260810 06:44:07.821347 19357 raft_consensus.cc:2454] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53d6b2f9b88a460b8f6b6a1517799857 in term 0.
I20260810 06:44:07.821347 19553 raft_consensus.cc:2454] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53d6b2f9b88a460b8f6b6a1517799857 in term 0.
I20260810 06:44:07.821902 19047 leader_election.cc:302] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 270dd939658740e89a862ebbf6f04be5, 53d6b2f9b88a460b8f6b6a1517799857; no voters:
I20260810 06:44:07.822295 19663 raft_consensus.cc:2781] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:44:07.822423 19663 raft_consensus.cc:482] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:07.822541 19663 raft_consensus.cc:3037] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:07.825165 19663 raft_consensus.cc:504] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } }
I20260810 06:44:07.825726 19663 leader_election.cc:288] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [CANDIDATE]: Term 1 election: Requested vote from peers 270dd939658740e89a862ebbf6f04be5 (127.10.242.3:46529), d39ee393a2ea432eb35fa88fa16bd644 (127.10.242.2:36171)
I20260810 06:44:07.826121 19553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ace20e21ea48439dd011a886f00853" candidate_uuid: "53d6b2f9b88a460b8f6b6a1517799857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "270dd939658740e89a862ebbf6f04be5"
I20260810 06:44:07.826244 19357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ace20e21ea48439dd011a886f00853" candidate_uuid: "53d6b2f9b88a460b8f6b6a1517799857" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d39ee393a2ea432eb35fa88fa16bd644"
I20260810 06:44:07.826349 19553 raft_consensus.cc:3037] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:07.826452 19357 raft_consensus.cc:3037] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:07.828596 19553 raft_consensus.cc:2454] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53d6b2f9b88a460b8f6b6a1517799857 in term 1.
I20260810 06:44:07.828656 19357 raft_consensus.cc:2454] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53d6b2f9b88a460b8f6b6a1517799857 in term 1.
I20260810 06:44:07.828953 19047 leader_election.cc:302] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 270dd939658740e89a862ebbf6f04be5, 53d6b2f9b88a460b8f6b6a1517799857; no voters:
I20260810 06:44:07.829217 19663 raft_consensus.cc:2781] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:07.829674 19663 raft_consensus.cc:686] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 1 LEADER]: Becoming Leader. State: Replica: 53d6b2f9b88a460b8f6b6a1517799857, State: Running, Role: LEADER
I20260810 06:44:07.830123 19663 consensus_queue.cc:237] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } }
I20260810 06:44:07.834952 18923 catalog_manager.cc:5495] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53d6b2f9b88a460b8f6b6a1517799857 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "53d6b2f9b88a460b8f6b6a1517799857" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53d6b2f9b88a460b8f6b6a1517799857" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 } health_report { overall_health: UNKNOWN } } }
W20260810 06:44:07.849725 19218 tablet.cc:2369] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:44:07.852980 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.850879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.10.242.1:45999)
0810 06:44:07.851049 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.851055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.851179 (+ 124us) server_negotiation.cc:407] Connection header received
0810 06:44:07.851397 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.851401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.851479 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.851562 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.852398 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.852404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.852409 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.852427 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.852457 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.852468 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.852563 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.852662 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.852720 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:44:07.860775 19357 raft_consensus.cc:1261] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [term 1 FOLLOWER]: Refusing update from remote peer 53d6b2f9b88a460b8f6b6a1517799857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:07.860941 19553 raft_consensus.cc:1261] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [term 1 FOLLOWER]: Refusing update from remote peer 53d6b2f9b88a460b8f6b6a1517799857: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:07.861670 19669 consensus_queue.cc:1035] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d39ee393a2ea432eb35fa88fa16bd644" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:07.861985 19663 consensus_queue.cc:1035] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [LEADER]: Connected to new peer: Peer: permanent_uuid: "270dd939658740e89a862ebbf6f04be5" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:07.877691 19673 mvcc.cc:205] Tried to move back new op lower bound from 7316866039228178432 to 7316866039116365824. Current Snapshot: MvccSnapshot[applied={T|T < 7316866039228178432}]
I20260810 06:44:07.890385 18923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:07.890625 18923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:07.893736 18923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce8f2951da8b4b23943e4aad4bbc084e: Sending DeleteTablet for 3 replicas of tablet c8ace20e21ea48439dd011a886f00853
I20260810 06:44:07.894884 19123 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ace20e21ea48439dd011a886f00853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:07 UTC) from {username='slave'} at 127.0.0.1:40392
I20260810 06:44:07.894912 19524 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ace20e21ea48439dd011a886f00853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:07 UTC) from {username='slave'} at 127.0.0.1:44750
I20260810 06:44:07.895195 19326 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ace20e21ea48439dd011a886f00853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:07 UTC) from {username='slave'} at 127.0.0.1:38438
I20260810 06:44:07.897320 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 19021
I20260810 06:44:07.907466 19688 tablet_replica.cc:330] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857: stopping tablet replica
I20260810 06:44:07.908242 19689 tablet_replica.cc:330] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: stopping tablet replica
I20260810 06:44:07.908715 19689 raft_consensus.cc:2229] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:07.908830 19688 raft_consensus.cc:2229] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:44:07.908829 19687 tablet_replica.cc:330] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: stopping tablet replica
I20260810 06:44:07.909237 19689 raft_consensus.cc:2258] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:07.909947 19688 raft_consensus.cc:2258] T c8ace20e21ea48439dd011a886f00853 P 53d6b2f9b88a460b8f6b6a1517799857 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:07.910236 19643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:07.898694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.10.242.1:45999)
0810 06:44:07.898838 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:07.898844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:07.907647 (+ 8803us) server_negotiation.cc:407] Connection header received
0810 06:44:07.907825 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:07.907830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:07.907900 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:07.907994 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:07.909324 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:07.909330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:07.909335 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:07.909353 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:07.909387 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:07.909402 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:07.909516 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:07.909790 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 06:44:07.909849 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:44:07.910485 19687 raft_consensus.cc:2229] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:07.911170 19687 raft_consensus.cc:2258] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:07.911401 19689 ts_tablet_manager.cc:1905] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:07.911644 19209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:07.913640 19687 ts_tablet_manager.cc:1905] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:07.919003 19689 ts_tablet_manager.cc:1918] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:07.919163 19689 log.cc:1196] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/c8ace20e21ea48439dd011a886f00853
I20260810 06:44:07.919667 19689 ts_tablet_manager.cc:1939] T c8ace20e21ea48439dd011a886f00853 P d39ee393a2ea432eb35fa88fa16bd644: Deleting consensus metadata
I20260810 06:44:07.920471 19687 ts_tablet_manager.cc:1918] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:07.920630 19687 log.cc:1196] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/c8ace20e21ea48439dd011a886f00853
I20260810 06:44:07.921135 19687 ts_tablet_manager.cc:1939] T c8ace20e21ea48439dd011a886f00853 P 270dd939658740e89a862ebbf6f04be5: Deleting consensus metadata
I20260810 06:44:07.921470 18905 catalog_manager.cc:4841] TS d39ee393a2ea432eb35fa88fa16bd644 (127.10.242.2:36171): tablet c8ace20e21ea48439dd011a886f00853 (table test-table [id=b5396bfd8bb94967afbdc018c64650a8]) successfully deleted
I20260810 06:44:07.922544 18903 catalog_manager.cc:4841] TS 270dd939658740e89a862ebbf6f04be5 (127.10.242.3:46529): tablet c8ace20e21ea48439dd011a886f00853 (table test-table [id=b5396bfd8bb94967afbdc018c64650a8]) successfully deleted
I20260810 06:44:08.005931 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 19021
W20260810 06:44:08.032617 18905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.1:45999 (error 108)
I20260810 06:44:08.033105 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 19222
I20260810 06:44:08.034670 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.034146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45999 (local address 127.0.0.1:40420)
0810 06:44:08.034345 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:08.034493 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:44:08.034927 18905 catalog_manager.cc:4584] TS 53d6b2f9b88a460b8f6b6a1517799857 (127.10.242.1:45999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8ace20e21ea48439dd011a886f00853: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45999: connect: Connection refused (error 111)
I20260810 06:44:08.035810 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.033899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.10.242.2:36171)
0810 06:44:08.034057 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:08.034062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:08.034081 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:44:08.034246 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:08.034251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:08.034363 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:08.034477 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:08.035302 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:08.035307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:08.035310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:08.035325 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:08.035355 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:08.035366 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:08.035467 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:08.035530 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 06:44:08.035576 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:44:08.036460 19419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:08.088588 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.088103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45999 (local address 127.0.0.1:40422)
0810 06:44:08.088319 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:08.088415 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 06:44:08.089680 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.089318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45999 (local address 127.0.0.1:40426)
0810 06:44:08.089469 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:08.089554 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 06:44:08.123327 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 19222
I20260810 06:44:08.146306 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 19429
I20260810 06:44:08.149170 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.147208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.10.242.3:46529)
0810 06:44:08.147404 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:08.147410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:08.147430 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:44:08.147495 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:08.147501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:08.147601 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:08.147693 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:08.148507 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:08.148513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:08.148517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:08.148533 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:08.148567 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:08.148579 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:08.148692 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:08.148808 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 06:44:08.148873 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:44:08.149923 19619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:08.150570 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.149881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45999 (local address 127.0.0.1:40428)
0810 06:44:08.150078 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:08.150375 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:44:08.151716 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.151333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45999 (local address 127.0.0.1:40436)
0810 06:44:08.151477 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:08.151573 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 06:44:08.232472 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.231940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45999 (local address 127.0.0.1:40442)
0810 06:44:08.232184 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:08.232294 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 06:44:08.233577 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.233161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45999 (local address 127.0.0.1:40454)
0810 06:44:08.233375 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:08.233449 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 06:44:08.237154 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 19429
I20260810 06:44:08.261886 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 18875
I20260810 06:44:08.263096 18975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:08.391443 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 18875
2026-08-10T06:44:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2542 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 06:44:08.441757 11208 mini_oidc.cc:152] Starting JWKS server
I20260810 06:44:08.444867 11208 webserver.cc:460] Webserver started at https://127.0.0.1:36855/ using document root <none> and password file <none>
I20260810 06:44:08.445173 11208 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:44:08.445801 11208 webserver.cc:460] Webserver started at http://127.0.0.1:33777/ using document root <none> and password file <none>
2026-08-10T06:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:08Z Disabled control of system clock
I20260810 06:44:08.475814 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:33245
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:40623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:33245
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36855/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/testchainca.cert with env {}
W20260810 06:44:08.791316 19707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:08.791601 19707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:08.791745 19707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:08.798693 19707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:08.798795 19707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:08.798858 19707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:08.798934 19707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:08.798995 19707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:44:08.799079 19707 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:44:08.799134 19707 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36855/jwks.json
I20260810 06:44:08.808506 19707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:33245
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:33245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36855/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:08.809698 19707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:08.810948 19707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:08.822230 19716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:08.822130 19719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:08.822175 19717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:08.822991 19707 server_base.cc:1034] running on GCE node
I20260810 06:44:08.823761 19707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:08.825069 19707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:08.826233 19707 hybrid_clock.cc:648] HybridClock initialized: now 1786344248826185 us; error 42 us; skew 500 ppm
I20260810 06:44:08.826571 19707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:08.828780 19707 webserver.cc:460] Webserver started at http://127.10.242.62:44447/ using document root <none> and password file <none>
I20260810 06:44:08.829442 19707 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:08.829550 19707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:08.829849 19707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:08.832773 19707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "6872f69af3f842a6b6bde0ea3b4f5a49"
format_stamp: "Formatted at 2026-08-10 06:44:08 on dist-test-slave-zj8q"
server_key: "e8f511eacc092aa615fb709128d60df4"
server_key_iv: "69c131b91b41f797e5b5d0882dfb0994"
server_key_version: "encryptionkey@0"
I20260810 06:44:08.833554 19707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "6872f69af3f842a6b6bde0ea3b4f5a49"
format_stamp: "Formatted at 2026-08-10 06:44:08 on dist-test-slave-zj8q"
server_key: "e8f511eacc092aa615fb709128d60df4"
server_key_iv: "69c131b91b41f797e5b5d0882dfb0994"
server_key_version: "encryptionkey@0"
I20260810 06:44:08.838476 19707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 06:44:08.841738 19732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:08.842971 19707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:08.843133 19707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "6872f69af3f842a6b6bde0ea3b4f5a49"
format_stamp: "Formatted at 2026-08-10 06:44:08 on dist-test-slave-zj8q"
server_key: "e8f511eacc092aa615fb709128d60df4"
server_key_iv: "69c131b91b41f797e5b5d0882dfb0994"
server_key_version: "encryptionkey@0"
I20260810 06:44:08.843322 19707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:08.873180 19707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:08.873972 19707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:08.874284 19707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:08.891067 19707 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:33245
I20260810 06:44:08.891103 19815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:33245 every 8 connection(s)
I20260810 06:44:08.892292 19707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:08.895961 19817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:08.902415 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 19707
I20260810 06:44:08.902645 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:08.903411 11208 external_mini_cluster.cc:1428] Setting key c2df3bc0e623008c3fd15abb02fc27de
I20260810 06:44:08.906564 19817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49: Bootstrap starting.
I20260810 06:44:08.909716 19817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:08.910727 19817 log.cc:824] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:08.913597 19817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49: No bootstrap required, opened a new log
I20260810 06:44:08.916363 19820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.905247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33536 (local address 127.10.242.62:33245)
0810 06:44:08.906180 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:08.906196 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:08.906245 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:44:08.907937 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:08.907959 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:08.908391 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:08.908680 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:08.909310 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:08.910651 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:08.912234 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:08.912852 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:08.912978 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:08.914197 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:08.914207 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:08.914221 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:08.914721 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:08.914794 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:08.914815 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:08.914901 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:08.915150 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 06:44:08.915444 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":319,"threads_started":1}
I20260810 06:44:08.920140 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:44:08.922627 19817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6872f69af3f842a6b6bde0ea3b4f5a49" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33245 } }
I20260810 06:44:08.923060 19817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:08.923213 19817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6872f69af3f842a6b6bde0ea3b4f5a49, State: Initialized, Role: FOLLOWER
I20260810 06:44:08.923810 19817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6872f69af3f842a6b6bde0ea3b4f5a49" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33245 } }
I20260810 06:44:08.924082 19817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:08.924237 19817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:08.924417 19817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:08.926496 19817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6872f69af3f842a6b6bde0ea3b4f5a49" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33245 } }
I20260810 06:44:08.926910 19817 leader_election.cc:302] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6872f69af3f842a6b6bde0ea3b4f5a49; no voters:
I20260810 06:44:08.927409 19817 leader_election.cc:288] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:08.927531 19824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:08.928264 19824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [term 1 LEADER]: Becoming Leader. State: Replica: 6872f69af3f842a6b6bde0ea3b4f5a49, State: Running, Role: LEADER
I20260810 06:44:08.928745 19824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6872f69af3f842a6b6bde0ea3b4f5a49" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33245 } }
I20260810 06:44:08.929229 19817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:08.931022 19826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6872f69af3f842a6b6bde0ea3b4f5a49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6872f69af3f842a6b6bde0ea3b4f5a49" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33245 } } }
I20260810 06:44:08.931170 19828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6872f69af3f842a6b6bde0ea3b4f5a49. Latest consensus state: current_term: 1 leader_uuid: "6872f69af3f842a6b6bde0ea3b4f5a49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6872f69af3f842a6b6bde0ea3b4f5a49" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33245 } } }
I20260810 06:44:08.931514 19826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:08.931587 19828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:08.932492 19835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:08.937686 19835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:08.951462 19835 catalog_manager.cc:1347] Generated new cluster ID: 69e6bb3335f54cd88e4ff29a5988324e
I20260810 06:44:08.951576 19835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:08.964262 19835 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:08.965445 19835 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:08.975492 19835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6872f69af3f842a6b6bde0ea3b4f5a49: Generated new TSK 0
I20260810 06:44:08.976261 19835 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:44:09.011435 19865 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 06:44:09.011665 19865 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:44:09.012769 19820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:08.992464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33550 (local address 127.10.242.62:33245)
0810 06:44:08.992609 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:08.992613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:08.993132 (+ 519us) server_negotiation.cc:407] Connection header received
0810 06:44:08.993479 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:08.993483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:08.993549 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:08.993663 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:44:08.994232 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:08.994953 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:08.996342 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:08.996813 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:08.996927 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:08.997405 (+ 478us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:44:09.012133 (+ 14728us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:44:09.012468 (+ 335us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33550: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 06:44:09.012899 19820 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33550: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 06:44:09.014421 19861 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:08.992453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:33245 (local address 127.0.0.1:33550)
0810 06:44:08.993032 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:08.993048 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:08.993222 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:08.993711 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:08.993716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:08.993728 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:44:08.994057 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:08.994067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:08.995084 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:08.995088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:08.996202 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:08.996215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:08.996973 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:08.996980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:08.997112 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:08.997167 (+ 55us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:44:09.012406 (+ 15239us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:44:09.012731 (+ 325us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.242.62:33245: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":295,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.242.62:33245: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 06:44:09.016040 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 19707
I20260810 06:44:09.017150 19809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:09.109644 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 19707
2026-08-10T06:44:09Z chronyd exiting
I20260810 06:44:09.167400 11208 test_util.cc:175] -----------------------------------------------
I20260810 06:44:09.167546 11208 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344206489568-11208-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (730 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 06:44:09.171326 11208 mini_oidc.cc:152] Starting JWKS server
I20260810 06:44:09.173106 11208 webserver.cc:460] Webserver started at https://127.0.0.1:37723/ using document root <none> and password file <none>
I20260810 06:44:09.173327 11208 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:44:09.173945 11208 webserver.cc:460] Webserver started at http://127.0.0.1:44151/ using document root <none> and password file <none>
2026-08-10T06:44:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:09Z Disabled control of system clock
I20260810 06:44:09.203464 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:46807
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:38053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:46807
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37723/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/testchainca.cert with env {}
W20260810 06:44:09.518663 19875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:09.518959 19875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:09.519093 19875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:09.525952 19875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:09.526057 19875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:09.526122 19875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:09.526216 19875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:09.526273 19875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:44:09.526355 19875 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:44:09.526409 19875 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37723/jwks.json
I20260810 06:44:09.535719 19875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:38053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:46807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:46807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37723/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:09.536901 19875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:09.538127 19875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:09.549104 19880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:09.549233 19882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:09.549202 19884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:09.550071 19875 server_base.cc:1034] running on GCE node
I20260810 06:44:09.550859 19875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:09.552138 19875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:09.553334 19875 hybrid_clock.cc:648] HybridClock initialized: now 1786344249553294 us; error 34 us; skew 500 ppm
I20260810 06:44:09.553648 19875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:09.559792 19875 webserver.cc:460] Webserver started at http://127.10.242.62:40403/ using document root <none> and password file <none>
I20260810 06:44:09.560433 19875 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:09.560546 19875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:09.560858 19875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:09.563777 19875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "9e0284b9674c4e1a88f7c899eeb7e59c"
format_stamp: "Formatted at 2026-08-10 06:44:09 on dist-test-slave-zj8q"
server_key: "682fb93836b9efa3474dfc7c81428778"
server_key_iv: "af183f04d49f5c2dc75f89d3ba6bd6ac"
server_key_version: "encryptionkey@0"
I20260810 06:44:09.564566 19875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "9e0284b9674c4e1a88f7c899eeb7e59c"
format_stamp: "Formatted at 2026-08-10 06:44:09 on dist-test-slave-zj8q"
server_key: "682fb93836b9efa3474dfc7c81428778"
server_key_iv: "af183f04d49f5c2dc75f89d3ba6bd6ac"
server_key_version: "encryptionkey@0"
I20260810 06:44:09.569588 19875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:44:09.572878 19893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:09.574185 19875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:44:09.574352 19875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "9e0284b9674c4e1a88f7c899eeb7e59c"
format_stamp: "Formatted at 2026-08-10 06:44:09 on dist-test-slave-zj8q"
server_key: "682fb93836b9efa3474dfc7c81428778"
server_key_iv: "af183f04d49f5c2dc75f89d3ba6bd6ac"
server_key_version: "encryptionkey@0"
I20260810 06:44:09.574545 19875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:09.586704 19875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:09.587464 19875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:09.587751 19875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:09.604629 19875 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:46807
I20260810 06:44:09.604652 19982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:46807 every 8 connection(s)
I20260810 06:44:09.605926 19875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:09.609458 19983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:09.610742 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 19875
I20260810 06:44:09.610955 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:09.611814 11208 external_mini_cluster.cc:1428] Setting key 420593121c93c5896d67d656ab68ad52
I20260810 06:44:09.621974 19983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c: Bootstrap starting.
I20260810 06:44:09.624300 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:09.613736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37406 (local address 127.10.242.62:46807)
0810 06:44:09.614778 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:09.614788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:09.614837 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:44:09.616324 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:09.616346 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:09.616781 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:09.617119 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:09.617644 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:09.618951 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:09.620199 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:09.620851 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:09.621019 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:09.622114 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:09.622125 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:09.622143 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:09.622715 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:09.622771 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:09.622791 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:09.622896 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:09.623107 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 06:44:09.623405 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":374,"threads_started":1}
I20260810 06:44:09.625085 19983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:09.626118 19983 log.cc:824] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:09.627771 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:44:09.629179 19983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c: No bootstrap required, opened a new log
I20260810 06:44:09.636689 19983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0284b9674c4e1a88f7c899eeb7e59c" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46807 } }
I20260810 06:44:09.636992 19983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:09.637070 19983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0284b9674c4e1a88f7c899eeb7e59c, State: Initialized, Role: FOLLOWER
I20260810 06:44:09.637518 19983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0284b9674c4e1a88f7c899eeb7e59c" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46807 } }
I20260810 06:44:09.637722 19983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:09.637814 19983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:09.637931 19983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:09.640095 19983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0284b9674c4e1a88f7c899eeb7e59c" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46807 } }
I20260810 06:44:09.640507 19983 leader_election.cc:302] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e0284b9674c4e1a88f7c899eeb7e59c; no voters:
I20260810 06:44:09.641026 19983 leader_election.cc:288] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:09.641170 19990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:09.641811 19990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [term 1 LEADER]: Becoming Leader. State: Replica: 9e0284b9674c4e1a88f7c899eeb7e59c, State: Running, Role: LEADER
I20260810 06:44:09.642385 19990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0284b9674c4e1a88f7c899eeb7e59c" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46807 } }
I20260810 06:44:09.642915 19983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:09.644990 19992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e0284b9674c4e1a88f7c899eeb7e59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0284b9674c4e1a88f7c899eeb7e59c" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46807 } } }
I20260810 06:44:09.645097 19993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e0284b9674c4e1a88f7c899eeb7e59c. Latest consensus state: current_term: 1 leader_uuid: "9e0284b9674c4e1a88f7c899eeb7e59c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0284b9674c4e1a88f7c899eeb7e59c" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46807 } } }
I20260810 06:44:09.645414 19992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:09.645455 19993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:09.646268 19996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:09.651556 19996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:09.660113 19996 catalog_manager.cc:1347] Generated new cluster ID: 16ece322824b4cbd98852c524b8e81be
I20260810 06:44:09.660211 19996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:09.676841 19996 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:09.678110 19996 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:09.690249 19996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e0284b9674c4e1a88f7c899eeb7e59c: Generated new TSK 0
I20260810 06:44:09.691071 19996 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:44:09.727451 20030 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 06:44:09.727578 20030 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:44:09.728410 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:09.701532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37420 (local address 127.10.242.62:46807)
0810 06:44:09.701715 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:09.701723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:09.702233 (+ 510us) server_negotiation.cc:407] Connection header received
0810 06:44:09.702473 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:09.702477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:09.702542 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:09.702654 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:44:09.703214 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:09.703891 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:09.705896 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:09.706403 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:09.706494 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:09.707066 (+ 572us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:44:09.727903 (+ 20837us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:44:09.728188 (+ 285us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37420: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 06:44:09.728523 19988 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37420: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 06:44:09.728741 20028 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:09.701499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:46807 (local address 127.0.0.1:37420)
0810 06:44:09.702094 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:09.702114 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:09.702310 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:09.702693 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:09.702698 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:09.702711 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:44:09.703030 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:09.703044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:09.704034 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:09.704040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:09.705765 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:09.705781 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:09.706592 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:09.706598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:09.706792 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:09.706838 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:44:09.728099 (+ 21261us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:44:09.728352 (+ 253us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.242.62:46807: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37723/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":297,"threads_started":1}
I20260810 06:44:09.729954 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 19875
I20260810 06:44:09.731000 19976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:09.822300 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 19875
2026-08-10T06:44:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (709 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 06:44:09.880427 11208 mini_oidc.cc:152] Starting JWKS server
I20260810 06:44:09.882293 11208 webserver.cc:460] Webserver started at https://127.0.0.1:44791/ using document root <none> and password file <none>
I20260810 06:44:09.882526 11208 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:44:09.883112 11208 webserver.cc:460] Webserver started at http://127.0.0.1:42807/ using document root <none> and password file <none>
2026-08-10T06:44:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:09Z Disabled control of system clock
I20260810 06:44:09.913384 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34505
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:34505
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44791/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:44:10.233973 20042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:10.234334 20042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:10.234480 20042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:10.241721 20042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:10.241825 20042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:10.241884 20042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:10.241962 20042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:10.242022 20042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:44:10.242105 20042 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:44:10.242178 20042 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44791/jwks.json
I20260810 06:44:10.251407 20042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:34505
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44791/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:10.252661 20042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:10.254492 20042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:10.265549 20049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:10.265671 20055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:10.265709 20052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:10.266547 20042 server_base.cc:1034] running on GCE node
I20260810 06:44:10.267261 20042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:10.268533 20042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:10.269733 20042 hybrid_clock.cc:648] HybridClock initialized: now 1786344250269690 us; error 35 us; skew 500 ppm
I20260810 06:44:10.270051 20042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:10.272115 20042 webserver.cc:460] Webserver started at http://127.10.242.62:37273/ using document root <none> and password file <none>
I20260810 06:44:10.272747 20042 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:10.272840 20042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:10.273142 20042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:10.275888 20042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "d036d265f54c4195bf6536b952a8f4a9"
format_stamp: "Formatted at 2026-08-10 06:44:10 on dist-test-slave-zj8q"
server_key: "5143cde5c669da4e1938ae14544d7f5b"
server_key_iv: "7afe94eb0aceecfaf008136a02b59083"
server_key_version: "encryptionkey@0"
I20260810 06:44:10.276619 20042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "d036d265f54c4195bf6536b952a8f4a9"
format_stamp: "Formatted at 2026-08-10 06:44:10 on dist-test-slave-zj8q"
server_key: "5143cde5c669da4e1938ae14544d7f5b"
server_key_iv: "7afe94eb0aceecfaf008136a02b59083"
server_key_version: "encryptionkey@0"
I20260810 06:44:10.281457 20042 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 06:44:10.284723 20067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:10.285956 20042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:44:10.286109 20042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "d036d265f54c4195bf6536b952a8f4a9"
format_stamp: "Formatted at 2026-08-10 06:44:10 on dist-test-slave-zj8q"
server_key: "5143cde5c669da4e1938ae14544d7f5b"
server_key_iv: "7afe94eb0aceecfaf008136a02b59083"
server_key_version: "encryptionkey@0"
I20260810 06:44:10.286331 20042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:10.309990 20042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:10.310803 20042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:10.311085 20042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:10.328281 20042 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:34505
I20260810 06:44:10.328300 20143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:34505 every 8 connection(s)
I20260810 06:44:10.329523 20042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:10.330828 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 20042
I20260810 06:44:10.331018 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:10.331648 11208 external_mini_cluster.cc:1428] Setting key 7b69e7cfec43f0643312843e7e675571
I20260810 06:44:10.333581 20144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:10.346390 20147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:10.333445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.10.242.62:34505)
0810 06:44:10.334511 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:10.334527 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:10.334579 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:44:10.336558 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:10.336585 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:10.337163 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:10.337599 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:10.338100 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:10.340141 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:10.341448 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:10.342461 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:10.342645 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:10.343657 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:10.343667 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:10.343688 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:10.344393 (+ 705us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:10.344495 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:10.344521 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:10.344627 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:10.344819 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 06:44:10.345144 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":336,"threads_started":1}
I20260810 06:44:10.347518 20144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9: Bootstrap starting.
I20260810 06:44:10.350215 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:44:10.351076 20144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:10.352226 20144 log.cc:824] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:10.355047 20144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9: No bootstrap required, opened a new log
I20260810 06:44:10.361804 20144 raft_consensus.cc:348] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d036d265f54c4195bf6536b952a8f4a9" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34505 } }
I20260810 06:44:10.362100 20144 raft_consensus.cc:374] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:10.362196 20144 raft_consensus.cc:729] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d036d265f54c4195bf6536b952a8f4a9, State: Initialized, Role: FOLLOWER
I20260810 06:44:10.362709 20144 consensus_queue.cc:260] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d036d265f54c4195bf6536b952a8f4a9" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34505 } }
I20260810 06:44:10.362914 20144 raft_consensus.cc:388] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:10.363044 20144 raft_consensus.cc:482] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:10.363193 20144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:10.365314 20144 raft_consensus.cc:504] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d036d265f54c4195bf6536b952a8f4a9" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34505 } }
I20260810 06:44:10.365789 20144 leader_election.cc:302] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d036d265f54c4195bf6536b952a8f4a9; no voters:
I20260810 06:44:10.366338 20144 leader_election.cc:288] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:10.366456 20151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:10.367146 20151 raft_consensus.cc:686] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [term 1 LEADER]: Becoming Leader. State: Replica: d036d265f54c4195bf6536b952a8f4a9, State: Running, Role: LEADER
I20260810 06:44:10.367677 20151 consensus_queue.cc:237] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d036d265f54c4195bf6536b952a8f4a9" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34505 } }
I20260810 06:44:10.368153 20144 sys_catalog.cc:564] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:10.370491 20152 sys_catalog.cc:455] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d036d265f54c4195bf6536b952a8f4a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d036d265f54c4195bf6536b952a8f4a9" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34505 } } }
I20260810 06:44:10.370877 20152 sys_catalog.cc:458] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:10.371465 20153 sys_catalog.cc:455] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d036d265f54c4195bf6536b952a8f4a9. Latest consensus state: current_term: 1 leader_uuid: "d036d265f54c4195bf6536b952a8f4a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d036d265f54c4195bf6536b952a8f4a9" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34505 } } }
I20260810 06:44:10.371795 20153 sys_catalog.cc:458] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:10.373232 20165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:10.377223 20165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:10.384426 20165 catalog_manager.cc:1347] Generated new cluster ID: 719543d26a9d47b4b9de59e2c2e6cac5
I20260810 06:44:10.384528 20165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:10.391772 20165 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:10.392868 20165 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:10.406243 20165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d036d265f54c4195bf6536b952a8f4a9: Generated new TSK 0
I20260810 06:44:10.406977 20165 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:44:10.434381 20147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:10.426239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60268 (local address 127.10.242.62:34505)
0810 06:44:10.426445 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:10.426452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:10.426901 (+ 449us) server_negotiation.cc:407] Connection header received
0810 06:44:10.427121 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:10.427125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:10.427203 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:10.427323 (+ 120us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:44:10.427828 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:10.428546 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:10.430540 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:10.430971 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:10.431059 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:10.431653 (+ 594us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:44:10.433729 (+ 2076us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:44:10.434042 (+ 313us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60268: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 06:44:10.434492 20147 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60268: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 06:44:10.436585 20183 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:10.426095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34505 (local address 127.0.0.1:60268)
0810 06:44:10.426793 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:10.426818 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:10.426966 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:10.427364 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:10.427369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:10.427383 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:44:10.427692 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:10.427707 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:10.428672 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:10.428678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:10.430407 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:10.430423 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:10.431163 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:10.431170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:10.431370 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:10.431415 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:44:10.433945 (+ 2530us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:44:10.436063 (+ 2118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.242.62:34505: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":1828,"thread_start_us":407,"threads_started":1}
I20260810 06:44:10.437826 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 20042
I20260810 06:44:10.438928 20137 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 06:44:10.445658 20184 webserver.cc:533] Webserver: sslize failed:
E20260810 06:44:10.445755 20184 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:44:10.528895 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 20042
2026-08-10T06:44:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (710 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T06:44:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:10Z Disabled control of system clock
I20260810 06:44:10.620900 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:33363
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:43051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:33363
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/testchainca.cert with env {}
W20260810 06:44:10.938737 20190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:10.939036 20190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:10.939162 20190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:10.946417 20190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:10.946519 20190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:10.946584 20190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:10.946663 20190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:10.946722 20190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:44:10.946807 20190 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:44:10.946863 20190 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 06:44:10.956643 20190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:33363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:33363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:10.957904 20190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:10.959256 20190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:10.971132 20201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:10.971079 20206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:10.971581 20203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:10.971939 20190 server_base.cc:1034] running on GCE node
I20260810 06:44:10.972725 20190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:10.974110 20190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:10.975364 20190 hybrid_clock.cc:648] HybridClock initialized: now 1786344250975307 us; error 42 us; skew 500 ppm
I20260810 06:44:10.975733 20190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:10.977941 20190 webserver.cc:460] Webserver started at http://127.10.242.62:44387/ using document root <none> and password file <none>
I20260810 06:44:10.978663 20190 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:10.978765 20190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:10.979065 20190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:10.981901 20190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "d89452e03dde44918c8a5bbcd6e7e372"
format_stamp: "Formatted at 2026-08-10 06:44:10 on dist-test-slave-zj8q"
server_key: "a0906b0b9b57f3a5125e20d6287408e7"
server_key_iv: "1d05f5994ca057c3f5b811d81a2f0fbb"
server_key_version: "encryptionkey@0"
I20260810 06:44:10.982746 20190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "d89452e03dde44918c8a5bbcd6e7e372"
format_stamp: "Formatted at 2026-08-10 06:44:10 on dist-test-slave-zj8q"
server_key: "a0906b0b9b57f3a5125e20d6287408e7"
server_key_iv: "1d05f5994ca057c3f5b811d81a2f0fbb"
server_key_version: "encryptionkey@0"
I20260810 06:44:10.987660 20190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:44:10.990924 20218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:10.992249 20190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:44:10.992408 20190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "d89452e03dde44918c8a5bbcd6e7e372"
format_stamp: "Formatted at 2026-08-10 06:44:10 on dist-test-slave-zj8q"
server_key: "a0906b0b9b57f3a5125e20d6287408e7"
server_key_iv: "1d05f5994ca057c3f5b811d81a2f0fbb"
server_key_version: "encryptionkey@0"
I20260810 06:44:10.992599 20190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:11.024080 20190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:11.024914 20190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:11.025182 20190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:11.042615 20190 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:33363
I20260810 06:44:11.042658 20299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:33363 every 8 connection(s)
I20260810 06:44:11.043932 20190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:11.047710 20301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:11.048806 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 20190
I20260810 06:44:11.048983 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:11.049639 11208 external_mini_cluster.cc:1428] Setting key 8aba4121b17dd98f38740afc025e22cd
I20260810 06:44:11.061348 20301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372: Bootstrap starting.
I20260810 06:44:11.063046 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:11.051598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33772 (local address 127.10.242.62:33363)
0810 06:44:11.052602 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:11.052613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:11.052671 (+ 58us) server_negotiation.cc:407] Connection header received
0810 06:44:11.054233 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:11.054258 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:11.054736 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:11.055072 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:11.055669 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:11.056935 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:11.058743 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:11.059455 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:11.059581 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:11.060804 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:11.060811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:11.060824 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:11.061348 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:11.061422 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:11.061443 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:11.061546 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:11.061784 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 06:44:11.062083 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":348,"threads_started":1}
I20260810 06:44:11.064733 20301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:11.065888 20301 log.cc:824] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:11.066658 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:44:11.068590 20301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372: No bootstrap required, opened a new log
I20260810 06:44:11.075556 20301 raft_consensus.cc:348] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89452e03dde44918c8a5bbcd6e7e372" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33363 } }
I20260810 06:44:11.075881 20301 raft_consensus.cc:374] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:11.075973 20301 raft_consensus.cc:729] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d89452e03dde44918c8a5bbcd6e7e372, State: Initialized, Role: FOLLOWER
I20260810 06:44:11.076548 20301 consensus_queue.cc:260] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89452e03dde44918c8a5bbcd6e7e372" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33363 } }
I20260810 06:44:11.076758 20301 raft_consensus.cc:388] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:11.076882 20301 raft_consensus.cc:482] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:11.077044 20301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:11.079145 20301 raft_consensus.cc:504] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89452e03dde44918c8a5bbcd6e7e372" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33363 } }
I20260810 06:44:11.079607 20301 leader_election.cc:302] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d89452e03dde44918c8a5bbcd6e7e372; no voters:
I20260810 06:44:11.080216 20301 leader_election.cc:288] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:11.080317 20313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:11.081118 20313 raft_consensus.cc:686] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [term 1 LEADER]: Becoming Leader. State: Replica: d89452e03dde44918c8a5bbcd6e7e372, State: Running, Role: LEADER
I20260810 06:44:11.081573 20313 consensus_queue.cc:237] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89452e03dde44918c8a5bbcd6e7e372" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33363 } }
I20260810 06:44:11.082207 20301 sys_catalog.cc:564] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:11.084405 20315 sys_catalog.cc:455] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d89452e03dde44918c8a5bbcd6e7e372" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89452e03dde44918c8a5bbcd6e7e372" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33363 } } }
I20260810 06:44:11.084401 20320 sys_catalog.cc:455] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d89452e03dde44918c8a5bbcd6e7e372. Latest consensus state: current_term: 1 leader_uuid: "d89452e03dde44918c8a5bbcd6e7e372" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d89452e03dde44918c8a5bbcd6e7e372" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33363 } } }
I20260810 06:44:11.084915 20320 sys_catalog.cc:458] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:11.084915 20315 sys_catalog.cc:458] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:11.087695 20325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:11.093170 20325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:11.104537 20325 catalog_manager.cc:1347] Generated new cluster ID: 35fc3602a7a44935b0c9e3200c3b6d5e
I20260810 06:44:11.104660 20325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:11.105875 20346 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:44:11.112371 20325 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:11.114126 20325 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:11.127628 20325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d89452e03dde44918c8a5bbcd6e7e372: Generated new TSK 0
I20260810 06:44:11.128471 20325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:44:11.167879 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:11.153334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33780 (local address 127.10.242.62:33363)
0810 06:44:11.153529 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:11.153535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:11.153987 (+ 452us) server_negotiation.cc:407] Connection header received
0810 06:44:11.154217 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:11.154224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:11.154315 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:11.154480 (+ 165us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:44:11.154916 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:11.156048 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:11.157434 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:11.158130 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:11.158292 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:11.158767 (+ 475us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:44:11.167182 (+ 8415us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:44:11.167558 (+ 376us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33780: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:44:11.167968 20362 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:11.153291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:33363 (local address 127.0.0.1:33780)
0810 06:44:11.153892 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:11.153908 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:11.154028 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:11.154516 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:11.154520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:11.154530 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:44:11.154777 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:11.154786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:11.156206 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:11.156211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:11.157296 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:11.157308 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:11.158320 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:11.158324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:11.158467 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:11.158499 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:44:11.167434 (+ 8935us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:44:11.167637 (+ 203us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.242.62:33363: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":299,"threads_started":1}
W20260810 06:44:11.168017 20311 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33780: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260810 06:44:11.169795 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 20190
I20260810 06:44:11.170924 20292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:11.263662 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 20190
2026-08-10T06:44:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (713 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (405 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (408 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:44:12 dist-test-slave-zj8q krb5kdc[20382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:44:12 dist-test-slave-zj8q krb5kdc[20382](info): set up 2 sockets
Aug 10 06:44:12 dist-test-slave-zj8q krb5kdc[20382](info): commencing operation
krb5kdc: starting...
W20260810 06:44:13.291481 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:44:13 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:44:13.507419 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:42835
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:42835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:13.825196 20399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:13.825476 20399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:13.825613 20399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:13.833014 20399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:13.833112 20399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:13.833191 20399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:13.833267 20399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:13.833339 20399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:13.843066 20399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:42835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:42835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:13.844376 20399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:13.845693 20399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:13.857000 20408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:13.857013 20412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:13.857096 20409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:13.857987 20399 server_base.cc:1034] running on GCE node
I20260810 06:44:13.858741 20399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:13.860024 20399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:13.861227 20399 hybrid_clock.cc:648] HybridClock initialized: now 1786344253861183 us; error 41 us; skew 500 ppm
I20260810 06:44:13.861538 20399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:13 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344253, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:13.864427 20399 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:44:13.865839 20399 webserver.cc:460] Webserver started at http://127.10.242.62:40171/ using document root <none> and password file <none>
I20260810 06:44:13.866519 20399 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:13.866640 20399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:13.866947 20399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:13.869805 20399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "c859b9e81f1a4943901ce781c425c461"
format_stamp: "Formatted at 2026-08-10 06:44:13 on dist-test-slave-zj8q"
server_key: "0f967e2b2dc89d2fce175d1a4292dda1"
server_key_iv: "533e342f8e70a2720c4c7594a2ef29c3"
server_key_version: "encryptionkey@0"
I20260810 06:44:13.870600 20399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "c859b9e81f1a4943901ce781c425c461"
format_stamp: "Formatted at 2026-08-10 06:44:13 on dist-test-slave-zj8q"
server_key: "0f967e2b2dc89d2fce175d1a4292dda1"
server_key_iv: "533e342f8e70a2720c4c7594a2ef29c3"
server_key_version: "encryptionkey@0"
I20260810 06:44:13.875347 20399 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 06:44:13.878522 20420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:13.879714 20399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:44:13.879868 20399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "c859b9e81f1a4943901ce781c425c461"
format_stamp: "Formatted at 2026-08-10 06:44:13 on dist-test-slave-zj8q"
server_key: "0f967e2b2dc89d2fce175d1a4292dda1"
server_key_iv: "533e342f8e70a2720c4c7594a2ef29c3"
server_key_version: "encryptionkey@0"
I20260810 06:44:13.880035 20399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:13.906278 20399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:13.909955 20399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:13.910236 20399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:13.927277 20399 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:42835
I20260810 06:44:13.927311 20500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:42835 every 8 connection(s)
I20260810 06:44:13.928532 20399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:13.932034 20501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:13.935585 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 20399
I20260810 06:44:13.935807 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:13.936611 11208 external_mini_cluster.cc:1428] Setting key 25bc540107e2b705e43d773068b8f78b
I20260810 06:44:13.944360 20501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461: Bootstrap starting.
I20260810 06:44:13.947294 20501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:13.948536 20501 log.cc:824] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:44:13 dist-test-slave-zj8q krb5kdc[20382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:13.951241 20501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461: No bootstrap required, opened a new log
I20260810 06:44:13.958119 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:13.938675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35432 (local address 127.10.242.62:42835)
0810 06:44:13.939416 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:13.939427 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:13.939470 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:44:13.941081 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:13.941105 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:13.941497 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:13.941867 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:13.943335 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:13.944650 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:13.945890 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:13.946590 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:13.946774 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:13.951091 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:13.951098 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:13.951111 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:13.951160 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:13.954191 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:13.954961 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:13.954968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:13.954974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:13.955156 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:13.955530 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:13.955534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:13.955536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:13.956047 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:13.956385 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:13.956765 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 06:44:13.957151 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":326,"threads_started":1}
I20260810 06:44:13.959218 20501 raft_consensus.cc:348] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c859b9e81f1a4943901ce781c425c461" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42835 } }
I20260810 06:44:13.959647 20501 raft_consensus.cc:374] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:13.959758 20501 raft_consensus.cc:729] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c859b9e81f1a4943901ce781c425c461, State: Initialized, Role: FOLLOWER
I20260810 06:44:13.960337 20501 consensus_queue.cc:260] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c859b9e81f1a4943901ce781c425c461" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42835 } }
I20260810 06:44:13.960564 20501 raft_consensus.cc:388] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:13.960707 20501 raft_consensus.cc:482] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:13.960893 20501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:13.963155 20501 raft_consensus.cc:504] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c859b9e81f1a4943901ce781c425c461" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42835 } }
I20260810 06:44:13.963588 20501 leader_election.cc:302] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c859b9e81f1a4943901ce781c425c461; no voters:
I20260810 06:44:13.964126 20501 leader_election.cc:288] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:13.964251 20509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:13.964911 20509 raft_consensus.cc:686] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [term 1 LEADER]: Becoming Leader. State: Replica: c859b9e81f1a4943901ce781c425c461, State: Running, Role: LEADER
I20260810 06:44:13.965454 20509 consensus_queue.cc:237] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c859b9e81f1a4943901ce781c425c461" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42835 } }
I20260810 06:44:13.966034 20501 sys_catalog.cc:564] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:13.968101 20511 sys_catalog.cc:455] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c859b9e81f1a4943901ce781c425c461. Latest consensus state: current_term: 1 leader_uuid: "c859b9e81f1a4943901ce781c425c461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c859b9e81f1a4943901ce781c425c461" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42835 } } }
I20260810 06:44:13.968209 20510 sys_catalog.cc:455] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c859b9e81f1a4943901ce781c425c461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c859b9e81f1a4943901ce781c425c461" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42835 } } }
I20260810 06:44:13.968518 20511 sys_catalog.cc:458] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:13.968541 20510 sys_catalog.cc:458] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:13.970326 20526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:13.975088 20526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:13.983320 20526 catalog_manager.cc:1347] Generated new cluster ID: ff4889be722e48a7b00f1ed2254f27b5
I20260810 06:44:13.983417 20526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:14.000651 20526 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:14.001840 20526 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:14.010854 20526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c859b9e81f1a4943901ce781c425c461: Generated new TSK 0
I20260810 06:44:14.011596 20526 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:44:14.079687 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42835
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:14.421872 20549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:14.422132 20549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:14.422271 20549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:14.429220 20549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:14.429315 20549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:14.429427 20549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:44:14.438967 20549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:14.440155 20549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:14.441381 20549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:14.453804 20557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:14.454241 20558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:14.454397 20562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:14.454761 20549 server_base.cc:1034] running on GCE node
I20260810 06:44:14.455399 20549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:14.456251 20549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:14.457445 20549 hybrid_clock.cc:648] HybridClock initialized: now 1786344254457401 us; error 35 us; skew 500 ppm
I20260810 06:44:14.457723 20549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:14 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344254, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:14.460817 20549 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:44:14.462342 20549 webserver.cc:460] Webserver started at http://127.10.242.1:33695/ using document root <none> and password file <none>
I20260810 06:44:14.463047 20549 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:14.463173 20549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:14.463507 20549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:14.466663 20549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "450cfe781f924e61b69146688d75ba33"
format_stamp: "Formatted at 2026-08-10 06:44:14 on dist-test-slave-zj8q"
server_key: "3ad109f0111c7f16bd4bc17dc1370c8c"
server_key_iv: "ce7bb103a0df0eb34184c94dd8eb2ffa"
server_key_version: "encryptionkey@0"
I20260810 06:44:14.467460 20549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "450cfe781f924e61b69146688d75ba33"
format_stamp: "Formatted at 2026-08-10 06:44:14 on dist-test-slave-zj8q"
server_key: "3ad109f0111c7f16bd4bc17dc1370c8c"
server_key_iv: "ce7bb103a0df0eb34184c94dd8eb2ffa"
server_key_version: "encryptionkey@0"
I20260810 06:44:14.472659 20549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:44:14.476092 20576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:14.477321 20549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:44:14.477473 20549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "450cfe781f924e61b69146688d75ba33"
format_stamp: "Formatted at 2026-08-10 06:44:14 on dist-test-slave-zj8q"
server_key: "3ad109f0111c7f16bd4bc17dc1370c8c"
server_key_iv: "ce7bb103a0df0eb34184c94dd8eb2ffa"
server_key_version: "encryptionkey@0"
I20260810 06:44:14.477646 20549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:14.505780 20549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:14.509620 20549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:14.509881 20549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:14.510902 20549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:14.512159 20549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:14.512229 20549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:14.512315 20549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:14.512357 20549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:14.540794 20549 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:46859
I20260810 06:44:14.540838 20764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:46859 every 8 connection(s)
I20260810 06:44:14.541986 20549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:14.545171 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 20549
I20260810 06:44:14.545399 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:44:14.546247 11208 external_mini_cluster.cc:1428] Setting key 10fb23da3b36553c9761eb57eb1d26a6
Aug 10 06:44:14 dist-test-slave-zj8q krb5kdc[20382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344254, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:14.567222 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:14.544382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:36177 (local address 127.10.242.62:42835)
0810 06:44:14.544799 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:14.544804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:14.545879 (+ 1075us) server_negotiation.cc:407] Connection header received
0810 06:44:14.547839 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:14.547846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:14.547932 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:14.548044 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:14.549949 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:14.550685 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:14.552668 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:14.553050 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:14.553146 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:14.557032 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:14.557036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:14.557040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:14.557070 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:14.563975 (+ 6905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:14.564829 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:14.564833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:14.564835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:14.564951 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:14.565434 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:14.565437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:14.565439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:14.565705 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:14.565823 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:14.566744 (+ 921us) server_negotiation.cc:299] Negotiation successful
0810 06:44:14.566923 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":241,"threads_started":1}
I20260810 06:44:14.568699 20777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:14.544879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42835 (local address 127.10.242.1:36177)
0810 06:44:14.545722 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:14.545760 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:14.547555 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:14.548306 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:14.548316 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:14.549033 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:14.549777 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:14.549803 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:14.550828 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:14.550833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:14.552545 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:14.552559 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:14.553258 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:14.553266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:14.553514 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:14.554194 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:14.554225 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:14.556820 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:14.564127 (+ 7307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:14.564137 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:14.564153 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:14.564729 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:14.565047 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:14.565052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:14.565056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:14.565337 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:14.565824 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:14.565831 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:14.566310 (+ 479us) client_negotiation.cc:769] Sending connection context
0810 06:44:14.566672 (+ 362us) client_negotiation.cc:240] Negotiation successful
0810 06:44:14.567238 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":358,"threads_started":1}
I20260810 06:44:14.570315 20770 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42835
I20260810 06:44:14.570613 20770 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
I20260810 06:44:14.571336 20770 heartbeater.cc:511] Master 127.10.242.62:42835 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
I20260810 06:44:14.573479 20452 ts_manager.cc:194] Registered new tserver with Master: 450cfe781f924e61b69146688d75ba33 (127.10.242.1:46859)
I20260810 06:44:14.575460 20452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:36177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:44:14.599249 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42835
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:14.913311 20780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:14.913573 20780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:14.913694 20780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:14.920630 20780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:14.920739 20780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:14.920884 20780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:14.929957 20780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:14.931038 20780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:14.932230 20780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:14.944510 20787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:14.944553 20789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:14.944577 20786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:14.945291 20780 server_base.cc:1034] running on GCE node
I20260810 06:44:14.945873 20780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:14.946758 20780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:14.947948 20780 hybrid_clock.cc:648] HybridClock initialized: now 1786344254947910 us; error 30 us; skew 500 ppm
I20260810 06:44:14.948220 20780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:14 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344254, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:14.951120 20780 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:44:14.952404 20780 webserver.cc:460] Webserver started at http://127.10.242.2:37183/ using document root <none> and password file <none>
I20260810 06:44:14.953032 20780 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:14.953130 20780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:14.953379 20780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:14.956193 20780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "11719d025031422085313e1c5606d7bb"
format_stamp: "Formatted at 2026-08-10 06:44:14 on dist-test-slave-zj8q"
server_key: "14a66f3817bb3cd8c0f59c83097cc330"
server_key_iv: "4652b9f85c852085994ae92ca13bb665"
server_key_version: "encryptionkey@0"
I20260810 06:44:14.956918 20780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "11719d025031422085313e1c5606d7bb"
format_stamp: "Formatted at 2026-08-10 06:44:14 on dist-test-slave-zj8q"
server_key: "14a66f3817bb3cd8c0f59c83097cc330"
server_key_iv: "4652b9f85c852085994ae92ca13bb665"
server_key_version: "encryptionkey@0"
I20260810 06:44:14.961578 20780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 06:44:14.964777 20802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:14.966005 20780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:44:14.966198 20780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "11719d025031422085313e1c5606d7bb"
format_stamp: "Formatted at 2026-08-10 06:44:14 on dist-test-slave-zj8q"
server_key: "14a66f3817bb3cd8c0f59c83097cc330"
server_key_iv: "4652b9f85c852085994ae92ca13bb665"
server_key_version: "encryptionkey@0"
I20260810 06:44:14.966374 20780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:14.978519 20780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:14.982216 20780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:14.982462 20780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:14.983402 20780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:14.984719 20780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:14.984798 20780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:14.984917 20780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:14.984970 20780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:15.014737 20780 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:38565
I20260810 06:44:15.014777 20985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:38565 every 8 connection(s)
I20260810 06:44:15.015938 20780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:15.017383 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 20780
I20260810 06:44:15.017556 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:44:15.018095 11208 external_mini_cluster.cc:1428] Setting key 3e8c45123d9116f2eadfb6a92356e91a
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344254, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:15.035626 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.018594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:43239 (local address 127.10.242.62:42835)
0810 06:44:15.018808 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.018814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.020114 (+ 1300us) server_negotiation.cc:407] Connection header received
0810 06:44:15.021896 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.021902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.021978 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.022063 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.023553 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.024261 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.025836 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.026310 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.026416 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.029818 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:15.029824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:15.029829 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:15.029869 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:15.032748 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.033402 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.033408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.033412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.033567 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.033935 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.033940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.033944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.034324 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:15.034492 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:15.035090 (+ 598us) server_negotiation.cc:299] Negotiation successful
0810 06:44:15.035314 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:44:15.036307 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.019138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42835 (local address 127.10.242.2:43239)
0810 06:44:15.019946 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:15.020006 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:15.021640 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:15.022331 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:15.022339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:15.022838 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:15.023389 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:15.023405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.024401 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.024405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:15.025692 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:15.025705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.026535 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.026543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:15.026747 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.027354 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:15.027377 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:15.029591 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:15.032875 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:15.032882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:15.032893 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:15.033282 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:15.033659 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:15.033662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:15.033664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:15.033838 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:15.034440 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:15.034446 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:15.034744 (+ 298us) client_negotiation.cc:769] Sending connection context
0810 06:44:15.034959 (+ 215us) client_negotiation.cc:240] Negotiation successful
0810 06:44:15.035302 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":296,"threads_started":1}
I20260810 06:44:15.037609 20990 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42835
I20260810 06:44:15.037922 20990 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:15.038817 20990 heartbeater.cc:511] Master 127.10.242.62:42835 requested a full tablet report, sending...
I20260810 06:44:15.040398 20452 ts_manager.cc:194] Registered new tserver with Master: 11719d025031422085313e1c5606d7bb (127.10.242.2:38565)
I20260810 06:44:15.041201 20452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:43239
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:44:15.071172 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42835
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:15.394820 20999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:15.395100 20999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:15.395236 20999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:15.402215 20999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:15.402320 20999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:15.402470 20999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:15.411902 20999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:15.413136 20999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:15.414431 20999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:15.427217 21018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:15.427260 21010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:15.427189 21012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:15.428184 20999 server_base.cc:1034] running on GCE node
I20260810 06:44:15.428785 20999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:15.429764 20999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:15.430970 20999 hybrid_clock.cc:648] HybridClock initialized: now 1786344255430923 us; error 33 us; skew 500 ppm
I20260810 06:44:15.431291 20999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:15.434350 20999 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:44:15.435701 20999 webserver.cc:460] Webserver started at http://127.10.242.3:34677/ using document root <none> and password file <none>
I20260810 06:44:15.436400 20999 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:15.436520 20999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:15.436821 20999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:15.439792 20999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "c7099e28d7014d3d8d7300972a2ae005"
format_stamp: "Formatted at 2026-08-10 06:44:15 on dist-test-slave-zj8q"
server_key: "4a8c071c31e677bdac8f1237d8783846"
server_key_iv: "c4f25b19566a4a069ce990231db3d234"
server_key_version: "encryptionkey@0"
I20260810 06:44:15.440588 20999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "c7099e28d7014d3d8d7300972a2ae005"
format_stamp: "Formatted at 2026-08-10 06:44:15 on dist-test-slave-zj8q"
server_key: "4a8c071c31e677bdac8f1237d8783846"
server_key_iv: "c4f25b19566a4a069ce990231db3d234"
server_key_version: "encryptionkey@0"
I20260810 06:44:15.445478 20999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:44:15.448823 21027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:15.450220 20999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:15.450371 20999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "c7099e28d7014d3d8d7300972a2ae005"
format_stamp: "Formatted at 2026-08-10 06:44:15 on dist-test-slave-zj8q"
server_key: "4a8c071c31e677bdac8f1237d8783846"
server_key_iv: "c4f25b19566a4a069ce990231db3d234"
server_key_version: "encryptionkey@0"
I20260810 06:44:15.450554 20999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:15.464383 20999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:15.468339 20999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:15.468608 20999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:15.469662 20999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:15.471138 20999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:15.471216 20999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:15.471331 20999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:15.471396 20999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:15.501524 20999 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:40029
I20260810 06:44:15.501582 21226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:40029 every 8 connection(s)
I20260810 06:44:15.502895 20999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:15.510589 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 20999
I20260810 06:44:15.510764 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:44:15.511319 11208 external_mini_cluster.cc:1428] Setting key 60a62d361bcc5d9786a5381df252126c
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:15.522061 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.505510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:54347 (local address 127.10.242.62:42835)
0810 06:44:15.505772 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.505778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.506891 (+ 1113us) server_negotiation.cc:407] Connection header received
0810 06:44:15.508609 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.508613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.508691 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.508777 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.510272 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.511431 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.513364 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.513790 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.513881 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.517233 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:15.517238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:15.517241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:15.517275 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:15.519645 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.520275 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.520279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.520281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.520403 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.520738 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.520740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.520742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.520982 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:15.521086 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:15.521657 (+ 571us) server_negotiation.cc:299] Negotiation successful
0810 06:44:15.521812 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260810 06:44:15.522992 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.505969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42835 (local address 127.10.242.3:54347)
0810 06:44:15.506734 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:15.506775 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:15.508375 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:15.508965 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:15.508974 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:15.509530 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:15.510088 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:15.510105 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.511602 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.511606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:15.513236 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:15.513249 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.514026 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.514034 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:15.514354 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.514935 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:15.514957 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:15.517050 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:15.519768 (+ 2718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:15.519776 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:15.519788 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:15.520188 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:15.520475 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:15.520478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:15.520480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:15.520659 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:15.521065 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:15.521071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:15.521369 (+ 298us) client_negotiation.cc:769] Sending connection context
0810 06:44:15.521569 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 06:44:15.521914 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":322,"threads_started":1}
I20260810 06:44:15.524305 21227 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42835
I20260810 06:44:15.524600 21227 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:15.525442 21227 heartbeater.cc:511] Master 127.10.242.62:42835 requested a full tablet report, sending...
I20260810 06:44:15.526943 20452 ts_manager.cc:194] Registered new tserver with Master: c7099e28d7014d3d8d7300972a2ae005 (127.10.242.3:40029)
I20260810 06:44:15.527701 20452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:54347
I20260810 06:44:15.538275 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/test-admin.keytab.
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:15.569001 11208 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 06:44:15.574730 21238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:15.571713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.10.242.1:46859)
0810 06:44:15.572915 (+ 1202us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.572922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.572950 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:44:15.573012 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.573018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.573233 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.573358 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.574300 (+ 942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55698: BlockingRecv error: recv got EOF from 127.0.0.1:55698 (error 108)
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":907,"threads_started":1}
W20260810 06:44:15.575172 11208 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5cc)
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:15.576778 11208 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:15.578111 11208 init.cc:295] Successfully reacquired a new kerberos TGT
I20260810 06:44:15.578919 20770 heartbeater.cc:503] Master 127.10.242.62:42835 was elected leader, sending a full tablet report...
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:44:15.594795 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.580191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55710 (local address 127.10.242.1:46859)
0810 06:44:15.580402 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.580408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.580898 (+ 490us) server_negotiation.cc:407] Connection header received
0810 06:44:15.580952 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.580955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.581049 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.581126 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.582285 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.583372 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.584733 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.585547 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.585659 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.589446 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:15.589460 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:15.589481 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:15.589531 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:15.592382 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.593002 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.593009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.593011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.593156 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.593455 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.593458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.593460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.593797 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:15.594013 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:15.594306 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:44:15.594521 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 06:44:15.600138 21238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:15.598605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.10.242.1:46859)
0810 06:44:15.598750 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.598753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.599082 (+ 329us) server_negotiation.cc:407] Connection header received
0810 06:44:15.599214 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.599217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.599272 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.599337 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.599941 (+ 604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55712: BlockingRecv error: recv got EOF from 127.0.0.1:55712 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 06:44:15.600368 11208 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:15.601799 11208 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:15.603080 11208 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:44:15.615971 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.604229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.10.242.1:46859)
0810 06:44:15.604392 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.604398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.604692 (+ 294us) server_negotiation.cc:407] Connection header received
0810 06:44:15.604832 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.604837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.604897 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.604969 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.605893 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.606622 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.607762 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.608183 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.608284 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.611474 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:15.611478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:15.611481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:15.611522 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:15.613710 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.614297 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.614302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.614305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.614420 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.614775 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.614780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.614784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.615028 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:15.615282 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:15.615544 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 06:44:15.615733 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 06:44:15.619808 21238 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:15.618270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.10.242.1:46859)
0810 06:44:15.618438 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.618444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.618697 (+ 253us) server_negotiation.cc:407] Connection header received
0810 06:44:15.618796 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.618800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.618863 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.618935 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.619592 (+ 657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55728: BlockingRecv error: recv got EOF from 127.0.0.1:55728 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 06:44:15.619956 11208 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344206489568-11208-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:15.621338 11208 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:15.622653 11208 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:44:15.636340 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.623950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.10.242.1:46859)
0810 06:44:15.624134 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.624140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.624372 (+ 232us) server_negotiation.cc:407] Connection header received
0810 06:44:15.624527 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.624531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.624587 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.624672 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.625677 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.626451 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.627663 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.628102 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.628211 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.631560 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:15.631564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:15.631567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:15.631598 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:15.633886 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.634534 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.634538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.634540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.634682 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.635028 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.635032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.635034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.635310 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:15.635589 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:15.635847 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 06:44:15.636059 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:44:15.637859 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 20549
I20260810 06:44:15.648849 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.638620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55738 (local address 127.10.242.1:46859)
0810 06:44:15.638758 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.638762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.639109 (+ 347us) server_negotiation.cc:407] Connection header received
0810 06:44:15.639238 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.639242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.639292 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.639363 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.640189 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.640846 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.641948 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.642396 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.642484 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.644386 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:15.644391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:15.644394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:15.644423 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:15.646722 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.647295 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.647299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.647301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.647420 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.647795 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.647798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.647800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.648030 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:15.648230 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:15.648443 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 06:44:15.648592 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:44:15.649399 20757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:15.732259 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 20549
I20260810 06:44:15.755724 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 20780
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:44:15.771710 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.756648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36868 (local address 127.10.242.2:38565)
0810 06:44:15.757214 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.757221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.757249 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:44:15.757329 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.757336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.757604 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.757737 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.758769 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.759809 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.761393 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.761948 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.762077 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.766216 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:15.766233 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:15.766255 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:15.766294 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:15.769162 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.769787 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.769794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.769796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.769914 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.770302 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.770307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.770310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.770670 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:15.770902 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:15.771175 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 06:44:15.771396 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":277,"threads_started":1}
I20260810 06:44:15.772936 20979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:15.854297 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 20780
I20260810 06:44:15.874742 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 20999
Aug 10 06:44:15 dist-test-slave-zj8q krb5kdc[20382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344255, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:44:15.889643 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:15.875607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.10.242.3:40029)
0810 06:44:15.876201 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:15.876208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:15.876238 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:44:15.876301 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:15.876307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:15.876513 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:15.876651 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:15.877583 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.878681 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.879882 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:15.880419 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:15.880521 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:15.884462 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:15.884480 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:15.884496 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:15.884598 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:15.887290 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.887895 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.887903 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.887905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.888035 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:15.888339 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:15.888342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:15.888345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:15.888699 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:15.888947 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:15.889122 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 06:44:15.889318 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":284,"threads_started":1}
I20260810 06:44:15.890967 21219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:15.971367 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 20999
I20260810 06:44:15.992364 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 20399
I20260810 06:44:15.993618 20494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:16.100764 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 20399
2026-08-10T06:44:16Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4026 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:44:16 dist-test-slave-zj8q krb5kdc[21270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:44:16 dist-test-slave-zj8q krb5kdc[21270](info): set up 2 sockets
Aug 10 06:44:16 dist-test-slave-zj8q krb5kdc[21270](info): commencing operation
krb5kdc: starting...
W20260810 06:44:17.328352 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:44:17 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:44:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.62.keytab.
Entry for principal oryx/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.62.keytab.
I20260810 06:44:17.535219 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34471
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.62.keytab
--principal=oryx/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:34471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:17.848690 21291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:17.848994 21291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:17.849135 21291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:17.856130 21291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:17.856235 21291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:17.856313 21291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:17.856387 21291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:17.856446 21291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:17.865923 21291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:34471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.62.keytab
--principal=oryx/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:17.867141 21291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:17.868399 21291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:17.879391 21301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:17.879441 21304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:17.879474 21299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:17.879925 21291 server_base.cc:1034] running on GCE node
I20260810 06:44:17.880645 21291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:17.881909 21291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:17.883106 21291 hybrid_clock.cc:648] HybridClock initialized: now 1786344257883060 us; error 32 us; skew 500 ppm
I20260810 06:44:17.883419 21291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:17 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:17.886307 21291 init.cc:377] Logged in from keytab as oryx/127.10.242.62@KRBTEST.COM (short username oryx)
I20260810 06:44:17.887653 21291 webserver.cc:460] Webserver started at http://127.10.242.62:36557/ using document root <none> and password file <none>
I20260810 06:44:17.888269 21291 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:17.888383 21291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:17.888671 21291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:17.891487 21291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "f6445ff13d22498e9dc52884364e67c0"
format_stamp: "Formatted at 2026-08-10 06:44:17 on dist-test-slave-zj8q"
server_key: "b6c9de25f34780aa890837335fb40430"
server_key_iv: "7af7d3c1fa8b36ee3e74c7cfb1da0d76"
server_key_version: "encryptionkey@0"
I20260810 06:44:17.892210 21291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "f6445ff13d22498e9dc52884364e67c0"
format_stamp: "Formatted at 2026-08-10 06:44:17 on dist-test-slave-zj8q"
server_key: "b6c9de25f34780aa890837335fb40430"
server_key_iv: "7af7d3c1fa8b36ee3e74c7cfb1da0d76"
server_key_version: "encryptionkey@0"
I20260810 06:44:17.896957 21291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:44:17.900127 21315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:17.901304 21291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:44:17.901463 21291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "f6445ff13d22498e9dc52884364e67c0"
format_stamp: "Formatted at 2026-08-10 06:44:17 on dist-test-slave-zj8q"
server_key: "b6c9de25f34780aa890837335fb40430"
server_key_iv: "7af7d3c1fa8b36ee3e74c7cfb1da0d76"
server_key_version: "encryptionkey@0"
I20260810 06:44:17.901640 21291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:17.932289 21291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:17.936156 21291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:17.936399 21291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:17.953251 21291 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:34471
I20260810 06:44:17.953289 21388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:34471 every 8 connection(s)
I20260810 06:44:17.954533 21291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:17.957958 21391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:17.961879 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 21291
I20260810 06:44:17.962056 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:17.962736 11208 external_mini_cluster.cc:1428] Setting key 9ce3f40fd96daa80a3221d19759e2e1a
I20260810 06:44:17.968443 21391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: Bootstrap starting.
I20260810 06:44:17.971858 21391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:17.972997 21391 log.cc:824] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:17.975571 21391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: No bootstrap required, opened a new log
Aug 10 06:44:17 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:17.984205 21391 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } }
I20260810 06:44:17.984607 21391 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:17.984635 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:17.964508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.10.242.62:34471)
0810 06:44:17.965244 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:17.965254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:17.965295 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:44:17.967094 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:17.967117 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:17.967466 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:17.967771 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:17.969593 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:17.970931 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:17.972597 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:17.973292 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:17.973446 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:17.977546 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:17.977554 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:17.977568 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:17.977606 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:17.980647 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:17.981437 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:17.981443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:17.981449 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:17.981619 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:17.981988 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:17.981991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:17.981993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:17.982553 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:17.982808 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:17.983221 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 06:44:17.983556 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":311,"threads_started":1}
I20260810 06:44:17.984732 21391 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6445ff13d22498e9dc52884364e67c0, State: Initialized, Role: FOLLOWER
I20260810 06:44:17.985329 21391 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } }
I20260810 06:44:17.985558 21391 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:17.985728 21391 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:17.985904 21391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:17.988535 21391 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } }
I20260810 06:44:17.988958 21391 leader_election.cc:302] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6445ff13d22498e9dc52884364e67c0; no voters:
I20260810 06:44:17.989462 21391 leader_election.cc:288] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:17.989570 21400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:17.990200 21400 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 1 LEADER]: Becoming Leader. State: Replica: f6445ff13d22498e9dc52884364e67c0, State: Running, Role: LEADER
I20260810 06:44:17.990701 21400 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } }
I20260810 06:44:17.991361 21391 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:17.993518 21401 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } }
I20260810 06:44:17.993878 21401 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:17.995694 21409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:17.995551 21402 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6445ff13d22498e9dc52884364e67c0. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } }
I20260810 06:44:17.995859 21402 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:18.000557 21409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:18.010221 21409 catalog_manager.cc:1347] Generated new cluster ID: 1e4c471737e54d9c8f57671acd4cdbcc
I20260810 06:44:18.010324 21409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:18.049403 21409 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:18.050876 21409 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:18.062717 21409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: Generated new TSK 0
I20260810 06:44:18.063560 21409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab.
Entry for principal oryx/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab.
I20260810 06:44:18.133836 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34471
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab
--principal=oryx/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:18.476467 21434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:18.476742 21434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:18.476876 21434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:18.483916 21434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:18.484026 21434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:18.484170 21434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:44:18.493790 21434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab
--principal=oryx/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:18.494993 21434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:18.496249 21434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:18.508642 21444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:18.508692 21445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:18.508726 21448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:18.509516 21434 server_base.cc:1034] running on GCE node
I20260810 06:44:18.510108 21434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:18.510974 21434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:18.512193 21434 hybrid_clock.cc:648] HybridClock initialized: now 1786344258512127 us; error 39 us; skew 500 ppm
I20260810 06:44:18.512485 21434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:18 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344258, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:18.515511 21434 init.cc:377] Logged in from keytab as oryx/127.10.242.1@KRBTEST.COM (short username oryx)
I20260810 06:44:18.516825 21434 webserver.cc:460] Webserver started at http://127.10.242.1:44901/ using document root <none> and password file <none>
I20260810 06:44:18.517453 21434 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:18.517567 21434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:18.517874 21434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:18.520655 21434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "4f53f4935ebb4d38a826be0c54c257df"
format_stamp: "Formatted at 2026-08-10 06:44:18 on dist-test-slave-zj8q"
server_key: "be2e99f1725a1afd2677b7247f2c519c"
server_key_iv: "27b3a6f307225de85fee307be89b1176"
server_key_version: "encryptionkey@0"
I20260810 06:44:18.521395 21434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "4f53f4935ebb4d38a826be0c54c257df"
format_stamp: "Formatted at 2026-08-10 06:44:18 on dist-test-slave-zj8q"
server_key: "be2e99f1725a1afd2677b7247f2c519c"
server_key_iv: "27b3a6f307225de85fee307be89b1176"
server_key_version: "encryptionkey@0"
I20260810 06:44:18.526093 21434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 06:44:18.529363 21459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:18.530588 21434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:44:18.530738 21434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "4f53f4935ebb4d38a826be0c54c257df"
format_stamp: "Formatted at 2026-08-10 06:44:18 on dist-test-slave-zj8q"
server_key: "be2e99f1725a1afd2677b7247f2c519c"
server_key_iv: "27b3a6f307225de85fee307be89b1176"
server_key_version: "encryptionkey@0"
I20260810 06:44:18.530906 21434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:18.553128 21434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:18.557204 21434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:18.557467 21434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:18.558444 21434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:18.559700 21434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:18.559779 21434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:18.559897 21434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:18.559952 21434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:18.590301 21434 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:45429
I20260810 06:44:18.590363 21638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:45429 every 8 connection(s)
I20260810 06:44:18.591843 21434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:18.599179 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 21434
I20260810 06:44:18.599354 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:44:18.599997 11208 external_mini_cluster.cc:1428] Setting key 9404b3db587030d70c5d9d0e55067bb6
Aug 10 06:44:18 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344258, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:18.611040 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:18.594273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:58825 (local address 127.10.242.62:34471)
0810 06:44:18.594713 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:18.594717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:18.595607 (+ 890us) server_negotiation.cc:407] Connection header received
0810 06:44:18.597373 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:18.597379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:18.597464 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:18.597568 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:18.599080 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:18.599856 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:18.601515 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:18.602029 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:18.602146 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:18.605546 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:18.605578 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:18.605581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:18.605613 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:18.608156 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:18.608846 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:18.608850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:18.608852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:18.608967 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:18.609374 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:18.609378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:18.609380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:18.609630 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:18.609771 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:18.610466 (+ 695us) server_negotiation.cc:299] Negotiation successful
0810 06:44:18.610700 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":213,"threads_started":1}
I20260810 06:44:18.611896 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:18.594719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.1:58825)
0810 06:44:18.595413 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:18.595449 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:18.597109 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:18.597805 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:18.597816 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:18.598367 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:18.598916 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:18.598934 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:18.600006 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:18.600010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:18.601373 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:18.601386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:18.602275 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:18.602283 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:18.602494 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:18.603090 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:18.603114 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:18.605340 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:18.608298 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:18.608307 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:18.608322 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:18.608727 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:18.609070 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:18.609075 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:18.609078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:18.609264 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:18.609753 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:18.609761 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:18.610068 (+ 307us) client_negotiation.cc:769] Sending connection context
0810 06:44:18.610330 (+ 262us) client_negotiation.cc:240] Negotiation successful
0810 06:44:18.610813 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":264,"threads_started":1}
I20260810 06:44:18.613458 21640 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34471
I20260810 06:44:18.613737 21640 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:18.614440 21640 heartbeater.cc:511] Master 127.10.242.62:34471 requested a full tablet report, sending...
I20260810 06:44:18.616614 21335 ts_manager.cc:194] Registered new tserver with Master: 4f53f4935ebb4d38a826be0c54c257df (127.10.242.1:45429)
I20260810 06:44:18.618502 21335 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:58825
WARNING: no policy specified for oryx/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab.
Entry for principal oryx/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab.
I20260810 06:44:18.652962 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34471
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab
--principal=oryx/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:18.970680 21655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:18.970932 21655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:18.971035 21655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:18.977909 21655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:18.978003 21655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:18.978118 21655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:18.987285 21655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab
--principal=oryx/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:18.988404 21655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:18.989598 21655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:19.001956 21663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:19.001956 21661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:19.002240 21670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:19.002772 21655 server_base.cc:1034] running on GCE node
I20260810 06:44:19.003355 21655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:19.004199 21655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:19.005393 21655 hybrid_clock.cc:648] HybridClock initialized: now 1786344259005348 us; error 33 us; skew 500 ppm
I20260810 06:44:19.005729 21655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:19 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344259, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:19.008672 21655 init.cc:377] Logged in from keytab as oryx/127.10.242.2@KRBTEST.COM (short username oryx)
I20260810 06:44:19.010128 21655 webserver.cc:460] Webserver started at http://127.10.242.2:36241/ using document root <none> and password file <none>
I20260810 06:44:19.010783 21655 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:19.010895 21655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:19.011173 21655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:19.013957 21655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "c8fb35a3e0384cdbad640fe4524933a8"
format_stamp: "Formatted at 2026-08-10 06:44:19 on dist-test-slave-zj8q"
server_key: "58f1ee180636ee9af9f633981409e2b1"
server_key_iv: "9f5fbb28a9cbac5797eecad2084eb984"
server_key_version: "encryptionkey@0"
I20260810 06:44:19.014737 21655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "c8fb35a3e0384cdbad640fe4524933a8"
format_stamp: "Formatted at 2026-08-10 06:44:19 on dist-test-slave-zj8q"
server_key: "58f1ee180636ee9af9f633981409e2b1"
server_key_iv: "9f5fbb28a9cbac5797eecad2084eb984"
server_key_version: "encryptionkey@0"
I20260810 06:44:19.019434 21655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:44:19.022547 21680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:19.023757 21655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 06:44:19.023897 21655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "c8fb35a3e0384cdbad640fe4524933a8"
format_stamp: "Formatted at 2026-08-10 06:44:19 on dist-test-slave-zj8q"
server_key: "58f1ee180636ee9af9f633981409e2b1"
server_key_iv: "9f5fbb28a9cbac5797eecad2084eb984"
server_key_version: "encryptionkey@0"
I20260810 06:44:19.024067 21655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:19.052361 21655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:19.057547 21655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:19.057868 21655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:19.059062 21655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:19.060808 21655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:19.060909 21655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:19.061019 21655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:19.061082 21655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:19.092473 21655 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:44043
I20260810 06:44:19.092494 21859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:44043 every 8 connection(s)
I20260810 06:44:19.094108 21655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:19.104187 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 21655
I20260810 06:44:19.104370 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:44:19.104818 11208 external_mini_cluster.cc:1428] Setting key 72dbc4322c1cc4b0d3dc19b23e23c89b
Aug 10 06:44:19 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344259, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:19.115975 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:19.096708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:39089 (local address 127.10.242.62:34471)
0810 06:44:19.096919 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:19.096926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:19.098300 (+ 1374us) server_negotiation.cc:407] Connection header received
0810 06:44:19.100842 (+ 2542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:19.100847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:19.100915 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:19.100996 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:19.102986 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:19.103688 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:19.106319 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:19.106918 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:19.107032 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:19.110303 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:19.110308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:19.110311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:19.110343 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:19.113050 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:19.113742 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:19.113746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:19.113749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:19.113872 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:19.114321 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:19.114324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:19.114326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:19.114586 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:19.114727 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:19.115418 (+ 691us) server_negotiation.cc:299] Negotiation successful
0810 06:44:19.115655 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:44:19.116947 21867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:19.097180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:39089)
0810 06:44:19.098088 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:19.098146 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:19.100565 (+ 2419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:19.101261 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:19.101273 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:19.101976 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:19.102818 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:19.102851 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:19.103844 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:19.103850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:19.106086 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:19.106104 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:19.107140 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:19.107149 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:19.107368 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:19.107972 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:19.107994 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:19.110051 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:19.113186 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:19.113195 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:19.113211 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:19.113615 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:19.113974 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:19.113979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:19.113983 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:19.114208 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:19.114717 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:19.114724 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:19.115040 (+ 316us) client_negotiation.cc:769] Sending connection context
0810 06:44:19.115276 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 06:44:19.115771 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":306,"threads_started":1}
I20260810 06:44:19.118431 21861 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34471
I20260810 06:44:19.118769 21861 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:19.119611 21861 heartbeater.cc:511] Master 127.10.242.62:34471 requested a full tablet report, sending...
I20260810 06:44:19.121258 21335 ts_manager.cc:194] Registered new tserver with Master: c8fb35a3e0384cdbad640fe4524933a8 (127.10.242.2:44043)
I20260810 06:44:19.122480 21335 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:39089
WARNING: no policy specified for oryx/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab.
Entry for principal oryx/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab.
I20260810 06:44:19.161270 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34471
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab
--principal=oryx/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:19.480434 21870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:19.480716 21870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:19.480840 21870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:19.487736 21870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:19.487851 21870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:19.487999 21870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:19.497390 21870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab
--principal=oryx/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:19.498692 21870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:19.499950 21870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:19.512239 21882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:19.512280 21885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:19.512307 21883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:19.513161 21870 server_base.cc:1034] running on GCE node
I20260810 06:44:19.513729 21870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:19.514603 21870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:19.515836 21870 hybrid_clock.cc:648] HybridClock initialized: now 1786344259515784 us; error 36 us; skew 500 ppm
I20260810 06:44:19.516142 21870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:19 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344259, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:19.519202 21870 init.cc:377] Logged in from keytab as oryx/127.10.242.3@KRBTEST.COM (short username oryx)
I20260810 06:44:19.520489 21870 webserver.cc:460] Webserver started at http://127.10.242.3:32985/ using document root <none> and password file <none>
I20260810 06:44:19.521107 21870 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:19.521225 21870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:19.521494 21870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:19.524235 21870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "a8ab00aad90243d29326e71b495f4638"
format_stamp: "Formatted at 2026-08-10 06:44:19 on dist-test-slave-zj8q"
server_key: "a40f3c822ca585b31995e22c837de5f0"
server_key_iv: "cfadefdb28956319f798d3080484f8e7"
server_key_version: "encryptionkey@0"
I20260810 06:44:19.524973 21870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "a8ab00aad90243d29326e71b495f4638"
format_stamp: "Formatted at 2026-08-10 06:44:19 on dist-test-slave-zj8q"
server_key: "a40f3c822ca585b31995e22c837de5f0"
server_key_iv: "cfadefdb28956319f798d3080484f8e7"
server_key_version: "encryptionkey@0"
I20260810 06:44:19.529841 21870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:44:19.533053 21897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:19.534337 21870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:19.534483 21870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "a8ab00aad90243d29326e71b495f4638"
format_stamp: "Formatted at 2026-08-10 06:44:19 on dist-test-slave-zj8q"
server_key: "a40f3c822ca585b31995e22c837de5f0"
server_key_iv: "cfadefdb28956319f798d3080484f8e7"
server_key_version: "encryptionkey@0"
I20260810 06:44:19.534648 21870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:19.545842 21870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:19.549305 21870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:19.549554 21870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:19.550436 21870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:19.551643 21870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:19.551728 21870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:19.551846 21870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:19.551898 21870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:19.580880 21870 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:38667
I20260810 06:44:19.580919 22081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:38667 every 8 connection(s)
I20260810 06:44:19.582123 21870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:19.583113 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 21870
I20260810 06:44:19.583317 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:44:19.583885 11208 external_mini_cluster.cc:1428] Setting key 8e2516a8068faf9933bfc806a957cfda
Aug 10 06:44:19 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344259, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:19.605929 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:19.584959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:35857 (local address 127.10.242.62:34471)
0810 06:44:19.585129 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:19.585134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:19.586496 (+ 1362us) server_negotiation.cc:407] Connection header received
0810 06:44:19.588271 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:19.588275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:19.588339 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:19.588413 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:19.589830 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:19.590948 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:19.592572 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:19.593209 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:19.593327 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:19.600684 (+ 7357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:19.600690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:19.600695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:19.600729 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:19.603023 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:19.603682 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:19.603687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:19.603691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:19.603816 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:19.604212 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:19.604217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:19.604221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:19.604500 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:19.604649 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:19.605343 (+ 694us) server_negotiation.cc:299] Negotiation successful
0810 06:44:19.605576 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:44:19.606808 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:19.585488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.3:35857)
0810 06:44:19.586334 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:19.586380 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:19.588040 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:19.588603 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:19.588610 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:19.589108 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:19.589678 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:19.589697 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:19.591104 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:19.591108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:19.592442 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:19.592454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:19.593411 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:19.593416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:19.593651 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:19.598355 (+ 4704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:19.598377 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:19.600487 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:19.603151 (+ 2664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:19.603160 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:19.603175 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:19.603570 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:19.603916 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:19.603922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:19.603926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:19.604106 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:19.604636 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:19.604643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:19.604980 (+ 337us) client_negotiation.cc:769] Sending connection context
0810 06:44:19.605216 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 06:44:19.605720 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":304,"threads_started":1}
I20260810 06:44:19.607976 22083 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34471
I20260810 06:44:19.608244 22083 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:19.608901 22083 heartbeater.cc:511] Master 127.10.242.62:34471 requested a full tablet report, sending...
I20260810 06:44:19.610335 21335 ts_manager.cc:194] Registered new tserver with Master: a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667)
I20260810 06:44:19.611105 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:44:19.611344 21335 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:35857
I20260810 06:44:19.621369 21640 heartbeater.cc:503] Master 127.10.242.62:34471 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.10.242.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.61.keytab.
Entry for principal oryx/127.10.242.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.61.keytab.
I20260810 06:44:19.662919 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.61:44177
--webserver_interface=127.10.242.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.61.keytab
--principal=oryx/127.10.242.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:34471,127.10.242.61:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:19.985556 22089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:19.985858 22089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:19.985992 22089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:19.993047 22089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:19.993151 22089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:19.993224 22089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:19.993299 22089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:19.993355 22089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:20.002892 22089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:34471,127.10.242.61:44177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.61.keytab
--principal=oryx/127.10.242.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.61:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.242.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
W20260810 06:44:20.003206 22089 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.242.62:34471,127.10.242.61:44177'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 06:44:20.004166 22089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:20.005505 22089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:20.017575 22101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:20.017604 22103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:20.018257 22106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:20.018594 22089 server_base.cc:1034] running on GCE node
I20260810 06:44:20.019565 22089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:20.020799 22089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:20.022006 22089 hybrid_clock.cc:648] HybridClock initialized: now 1786344260021922 us; error 64 us; skew 500 ppm
I20260810 06:44:20.022342 22089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344260, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:20.025321 22089 init.cc:377] Logged in from keytab as oryx/127.10.242.61@KRBTEST.COM (short username oryx)
I20260810 06:44:20.026695 22089 webserver.cc:460] Webserver started at http://127.10.242.61:34763/ using document root <none> and password file <none>
I20260810 06:44:20.027369 22089 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:20.027467 22089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:20.027755 22089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:20.030650 22089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data/instance:
uuid: "ab293efd178e4e7698d549af49df8d67"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a7730582d75d8f511597c66abbda047c"
server_key_iv: "95c38849ba75e53d36f2da66825be463"
server_key_version: "encryptionkey@0"
I20260810 06:44:20.031454 22089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal/instance:
uuid: "ab293efd178e4e7698d549af49df8d67"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a7730582d75d8f511597c66abbda047c"
server_key_iv: "95c38849ba75e53d36f2da66825be463"
server_key_version: "encryptionkey@0"
I20260810 06:44:20.036413 22089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:44:20.039844 22117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:20.041083 22089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:44:20.041245 22089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal
uuid: "ab293efd178e4e7698d549af49df8d67"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a7730582d75d8f511597c66abbda047c"
server_key_iv: "95c38849ba75e53d36f2da66825be463"
server_key_version: "encryptionkey@0"
I20260810 06:44:20.041435 22089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:20.061233 22089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:20.065204 22089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:20.065430 22089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:20.082324 22089 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.61:44177
I20260810 06:44:20.082360 22198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.61:44177 every 8 connection(s)
I20260810 06:44:20.083560 22089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
I20260810 06:44:20.087034 22200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:20.092360 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 22089
I20260810 06:44:20.093541 22200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344260, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.61@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.61@KRBTEST.COM
I20260810 06:44:20.112706 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.096913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48406 (local address 127.10.242.62:34471)
0810 06:44:20.097095 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.097101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.097912 (+ 811us) server_negotiation.cc:407] Connection header received
0810 06:44:20.098438 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.098444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.098519 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.098602 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.099811 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.100531 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.101984 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.102437 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.102547 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.106764 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.106770 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.106775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.106817 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.109421 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.110328 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.110331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.110334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.110460 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.110932 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.110936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.110939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.111364 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.111483 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.112208 (+ 725us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.112372 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:44:20.113874 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.096951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:48406)
0810 06:44:20.097777 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.097815 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.098220 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.098679 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.098686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.099304 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.099652 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.099667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.100663 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.100669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.101842 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.101855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.102656 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.102664 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.103481 (+ 817us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.104004 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.104029 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.106563 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.109545 (+ 2982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.109554 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.109564 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.110216 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.110566 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.110571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.110574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.110816 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.111474 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.111480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.111821 (+ 341us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.112114 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.112565 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":610,"spinlock_wait_cycles":3328,"thread_start_us":400,"threads_started":1}
I20260810 06:44:20.113881 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.094719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51544 (local address 127.10.242.61:44177)
0810 06:44:20.095494 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.095510 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.095562 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:44:20.097927 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.097954 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.098311 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.098492 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.099940 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.101084 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.102284 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.102841 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.102990 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.107543 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.107554 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.107568 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.107606 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.110627 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.111407 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.111414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.111418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.111570 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.111928 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.111932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.111935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.112305 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.112593 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.112897 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.113128 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":541,"spinlock_wait_cycles":49152,"thread_start_us":317,"threads_started":1}
I20260810 06:44:20.115654 22200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344260, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.61@KRBTEST.COM for oryx/127.10.242.61@KRBTEST.COM
I20260810 06:44:20.125566 21861 heartbeater.cc:503] Master 127.10.242.62:34471 was elected leader, sending a full tablet report...
I20260810 06:44:20.126546 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.116262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.0.0.1:51560)
0810 06:44:20.116432 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.116449 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.116543 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.116811 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.116814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.117092 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.117216 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.117223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.118087 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.118090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.118955 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.118966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.119562 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.119566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.119717 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.120122 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.120139 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.121873 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.124493 (+ 2620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.124497 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.124499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.124844 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.125119 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.125122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.125125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.125244 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.125752 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.125757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.125902 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.126012 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.126309 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 06:44:20.126820 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.116288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51560 (local address 127.10.242.61:44177)
0810 06:44:20.116433 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.116437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.116494 (+ 57us) server_negotiation.cc:407] Connection header received
0810 06:44:20.116632 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.116636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.116694 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.116782 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.117323 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.117969 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.119062 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.119449 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.119537 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.122028 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.122033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.122036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.122064 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.124384 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.124932 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.124935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.124938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.125045 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.125324 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.125327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.125329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.125604 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.125717 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.126130 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.126479 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":46}
I20260810 06:44:20.131340 22200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Bootstrap starting.
I20260810 06:44:20.134321 22200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:20.135275 22200 log.cc:824] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:20.137722 22200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: No bootstrap required, opened a new log
I20260810 06:44:20.144536 22200 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } }
I20260810 06:44:20.144819 22200 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:20.144886 22200 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab293efd178e4e7698d549af49df8d67, State: Initialized, Role: FOLLOWER
I20260810 06:44:20.145423 22200 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } }
I20260810 06:44:20.146225 22212 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } } }
I20260810 06:44:20.146528 22212 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:44:20.147140 22200 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:44:20.155198 22233 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:44:20.155311 22233 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:44:20.158449 22089 master_runner.cc:428] Detected that this master ab293efd178e4e7698d549af49df8d67 is joining an existing cluster
I20260810 06:44:20.158527 22089 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.242.61:44177
I20260810 06:44:20.177245 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.162173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.10.242.61:44177)
0810 06:44:20.162348 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.162357 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.163517 (+ 1160us) server_negotiation.cc:407] Connection header received
0810 06:44:20.163642 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.163647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.163708 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.163798 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.165072 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.165804 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.167544 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.168314 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.168442 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.170719 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.170725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.170729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.170757 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.174286 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.175066 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.175071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.175074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.175220 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.175640 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.175645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.175648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.176017 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.176159 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.176610 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.176837 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":40}
I20260810 06:44:20.178229 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.162104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.10.242.62:34471)
0810 06:44:20.162320 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.162324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.163115 (+ 791us) server_negotiation.cc:407] Connection header received
0810 06:44:20.163450 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.163456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.163544 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.163645 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.165200 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.166401 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.168919 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.169510 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.169610 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.172092 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.172099 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.172104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.172145 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.175272 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.175968 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.175974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.175977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.176137 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.176554 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.176558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.176561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.176900 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.177019 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.177650 (+ 631us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.177871 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:44:20.178224 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.162447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.0.0.1:51570)
0810 06:44:20.163438 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.163451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.163548 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.163949 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.163953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.164571 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.164947 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.164958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.165966 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.165969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.167391 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.167405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.168515 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.168522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.168753 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.169398 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.169420 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.170529 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.174419 (+ 3890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.174423 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.174427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.174945 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.175343 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.175347 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.175351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.175532 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.176225 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.176228 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.176376 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.177496 (+ 1120us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.177753 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":27,"thread_start_us":686,"threads_started":1}
I20260810 06:44:20.179183 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.162365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:48422)
0810 06:44:20.163009 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.163029 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.163245 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.163694 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.163697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.164400 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.164857 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.164871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.166543 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.166549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.168753 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.168767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.169914 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.169920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.170090 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.170688 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.170710 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.171878 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.175399 (+ 3521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.175403 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.175406 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.175833 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.176267 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.176271 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.176274 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.176430 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.177109 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.177113 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.177310 (+ 197us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.178465 (+ 1155us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.178696 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":649,"thread_start_us":370,"threads_started":1}
WARNING: no policy specified for oryx/127.10.242.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.60@KRBTEST.COM" created.
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.62@KRBTEST.COM for oryx/127.10.242.61@KRBTEST.COM
I20260810 06:44:20.205663 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.185109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.10.242.61:44177)
0810 06:44:20.185278 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.185283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.190831 (+ 5548us) server_negotiation.cc:407] Connection header received
0810 06:44:20.191272 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.191278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.191366 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.191465 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.193058 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.194138 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.196004 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.196738 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.196859 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.200322 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.200327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.200330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.200361 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.203000 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.203677 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.203682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.203685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.203813 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.204174 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.204178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.204180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.204451 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.204579 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.205082 (+ 503us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.205312 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:44:20.205789 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.184948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.0.0.1:51578)
0810 06:44:20.190691 (+ 5743us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.190739 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.190990 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.191562 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.191570 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.192603 (+ 1033us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.192886 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.192899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.194311 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.194317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.195833 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.195850 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.196929 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.196934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.197080 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.197690 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.197707 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.200142 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.203128 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.203141 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.203149 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.203562 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.203927 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.203932 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.203936 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.204068 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.204589 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.204598 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.204788 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.204953 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.205278 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5520,"thread_start_us":5368,"threads_started":1}
I20260810 06:44:20.207813 21335 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.242.61:44177: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ab293efd178e4e7698d549af49df8d67"
member_type: NON_VOTER
last_known_addr {
host: "127.10.242.61"
port: 44177
}
attrs {
promote: true
}
}
dest_uuid: "f6445ff13d22498e9dc52884364e67c0"
cas_config_opid_index: -1
I20260810 06:44:20.208861 21335 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } }
I20260810 06:44:20.224686 22162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:20.224303 22255 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ab293efd178e4e7698d549af49df8d67 (127.10.242.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } } }
I20260810 06:44:20.227948 22255 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.242.61:44177
I20260810 06:44:20.228572 22162 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Refusing update from remote peer f6445ff13d22498e9dc52884364e67c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 06:44:20.228905 22089 master.cc:537] Master@127.10.242.61:44177 shutting down...
I20260810 06:44:20.229082 22249 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } } }
I20260810 06:44:20.229385 22249 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:20.231526 22255 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:44:20.235273 21316 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.60.keytab.
Entry for principal oryx/127.10.242.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.60.keytab.
I20260810 06:44:20.237911 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.60:41639
--webserver_interface=127.10.242.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.60.keytab
--principal=oryx/127.10.242.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:34471,127.10.242.61:44177,127.10.242.60:41639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:44:20.245122 22089 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:20.245586 22089 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:20.245787 22089 tablet_replica.cc:330] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: stopping tablet replica
I20260810 06:44:20.256520 22089 master.cc:552] Master@127.10.242.61:44177 shutdown complete.
I20260810 06:44:20.256664 22089 master_runner.cc:305] Clearing existing system tablet
I20260810 06:44:20.257181 22089 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:20.257277 22089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:20.259526 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.259000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.0.0.1:51584)
0810 06:44:20.259203 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.259356 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260810 06:44:20.259903 21316 consensus_peers.cc:487] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 -> Peer ab293efd178e4e7698d549af49df8d67 (127.10.242.61:44177): Couldn't send request to peer ab293efd178e4e7698d549af49df8d67. Status: Network error: Client connection negotiation failed: client connection to 127.10.242.61:44177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:44:20.261320 22089 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:44:20.263406 22262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:20.263733 22089 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:20.264012 22089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:44:20.264096 22089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal
uuid: "ab293efd178e4e7698d549af49df8d67"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a7730582d75d8f511597c66abbda047c"
server_key_iv: "95c38849ba75e53d36f2da66825be463"
server_key_version: "encryptionkey@0"
I20260810 06:44:20.269137 22089 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:20.275928 22089 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:44:20.276087 22089 log.cc:1196] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 06:44:20.276497 22089 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Deleting consensus metadata
I20260810 06:44:20.276934 22089 master_runner.cc:315] Copying system tablet from 127.10.242.62:34471
I20260810 06:44:20.279441 22089 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: tablet copy: Beginning tablet copy session from remote peer at address 127.10.242.62:34471
I20260810 06:44:20.296348 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.280201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.10.242.62:34471)
0810 06:44:20.280369 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.280376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.282718 (+ 2342us) server_negotiation.cc:407] Connection header received
0810 06:44:20.282884 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.282890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.282978 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.283097 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.284090 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.285193 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.286940 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.287591 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.287723 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.289878 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.289886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.289891 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.289931 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.293458 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.294234 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.294239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.294243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.294387 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.294813 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.294817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.294821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.295177 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.295323 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.295754 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.296006 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:44:20.296365 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.280199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:48428)
0810 06:44:20.282572 (+ 2373us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.282599 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.282763 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.283134 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.283139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.283630 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.283918 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.283932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.285349 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.285354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.286795 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.286810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.287763 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.287768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.287951 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.288642 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.288667 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.289658 (+ 991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.293581 (+ 3923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.293586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.293590 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.294098 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.294506 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.294511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.294514 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.294699 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.295298 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.295304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.295487 (+ 183us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.295642 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.295933 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2228,"thread_start_us":2124,"threads_started":1}
I20260810 06:44:20.297634 21369 tablet_copy_service.cc:140] P f6445ff13d22498e9dc52884364e67c0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ab293efd178e4e7698d549af49df8d67 ({username='oryx', principal='oryx/127.10.242.61@KRBTEST.COM'} at 127.0.0.1:48428)
I20260810 06:44:20.297927 21369 tablet_copy_service.cc:161] P f6445ff13d22498e9dc52884364e67c0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ab293efd178e4e7698d549af49df8d67 at {username='oryx', principal='oryx/127.10.242.61@KRBTEST.COM'} at 127.0.0.1:48428: session id = ab293efd178e4e7698d549af49df8d67-00000000000000000000000000000000
I20260810 06:44:20.302448 21369 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:44:20.304662 22089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:44:20.311713 22089 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: tablet copy: Starting download of 0 data blocks...
I20260810 06:44:20.312024 22089 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: tablet copy: Starting download of 1 WAL segments...
I20260810 06:44:20.314709 22089 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:44:20.318936 21369 tablet_copy_service.cc:342] P f6445ff13d22498e9dc52884364e67c0: Request end of tablet copy session ab293efd178e4e7698d549af49df8d67-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.242.61@KRBTEST.COM'} at 127.0.0.1:48428
I20260810 06:44:20.319151 21369 tablet_copy_service.cc:433] P f6445ff13d22498e9dc52884364e67c0: ending tablet copy session ab293efd178e4e7698d549af49df8d67-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ab293efd178e4e7698d549af49df8d67
I20260810 06:44:20.334789 22089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:20.337435 22272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:20.339268 22273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:20.349944 22278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:20.350340 22089 server_base.cc:1034] running on GCE node
I20260810 06:44:20.350724 22089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:20.351403 22089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:20.352687 22089 hybrid_clock.cc:648] HybridClock initialized: now 1786344260352622 us; error 36 us; skew 500 ppm
I20260810 06:44:20.352974 22089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344260, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:20.355336 22089 init.cc:377] Logged in from keytab as oryx/127.10.242.61@KRBTEST.COM (short username oryx)
I20260810 06:44:20.356632 22089 webserver.cc:460] Webserver started at http://127.10.242.61:36829/ using document root <none> and password file <none>
I20260810 06:44:20.357062 22089 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:20.357224 22089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:20.360626 22089 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:44:20.362659 22287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:20.363091 22089 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:44:20.363283 22089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal
uuid: "ab293efd178e4e7698d549af49df8d67"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a7730582d75d8f511597c66abbda047c"
server_key_iv: "95c38849ba75e53d36f2da66825be463"
server_key_version: "encryptionkey@0"
I20260810 06:44:20.363540 22089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:20.382187 22089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:20.386533 22089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:20.416460 22089 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.61:44177
I20260810 06:44:20.416486 22371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.61:44177 every 8 connection(s)
I20260810 06:44:20.417512 22089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
I20260810 06:44:20.422572 22372 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:44:20.426486 22372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Bootstrap starting.
I20260810 06:44:20.443516 22372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:44:20.444006 22372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Bootstrap complete.
I20260810 06:44:20.445029 22372 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } }
I20260810 06:44:20.445307 22372 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ab293efd178e4e7698d549af49df8d67, State: Initialized, Role: LEARNER
I20260810 06:44:20.445577 22372 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } }
I20260810 06:44:20.446401 22377 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } } }
I20260810 06:44:20.446708 22372 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:20.446862 22377 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:44:20.458305 22399 catalog_manager.cc:1259] Loaded cluster ID: 1e4c471737e54d9c8f57671acd4cdbcc
I20260810 06:44:20.458411 22399 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: loading cluster ID for follower catalog manager: success
I20260810 06:44:20.461992 22399 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: acquiring CA information for follower catalog manager: success
I20260810 06:44:20.464501 22399 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:44:20.613710 22083 heartbeater.cc:503] Master 127.10.242.62:34471 was elected leader, sending a full tablet report...
W20260810 06:44:20.630731 22259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:20.631002 22259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:20.631143 22259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:20.638662 22259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:20.638782 22259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:20.638859 22259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:20.638937 22259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:20.638998 22259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:20.648767 22259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:34471,127.10.242.61:44177,127.10.242.60:41639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.60.keytab
--principal=oryx/127.10.242.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.60:41639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.242.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:20.650183 22259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:20.651467 22259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:20.662598 22409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:20.662566 22414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:20.662619 22410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:20.663527 22259 server_base.cc:1034] running on GCE node
I20260810 06:44:20.664243 22259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:20.665546 22259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:20.666751 22259 hybrid_clock.cc:648] HybridClock initialized: now 1786344260666707 us; error 38 us; skew 500 ppm
I20260810 06:44:20.667068 22259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344260, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:20.670020 22259 init.cc:377] Logged in from keytab as oryx/127.10.242.60@KRBTEST.COM (short username oryx)
I20260810 06:44:20.671365 22259 webserver.cc:460] Webserver started at http://127.10.242.60:45369/ using document root <none> and password file <none>
I20260810 06:44:20.672012 22259 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:20.672132 22259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:20.672411 22259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:20.675261 22259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data/instance:
uuid: "f77c30f7f2554020b85fa930af9287c5"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a05feb899ca981c28b4f0d5b28a6ae79"
server_key_iv: "69fbf6b749bf8ffa11bf5c3e00f121b7"
server_key_version: "encryptionkey@0"
I20260810 06:44:20.675997 22259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal/instance:
uuid: "f77c30f7f2554020b85fa930af9287c5"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a05feb899ca981c28b4f0d5b28a6ae79"
server_key_iv: "69fbf6b749bf8ffa11bf5c3e00f121b7"
server_key_version: "encryptionkey@0"
I20260810 06:44:20.680922 22259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:44:20.684128 22424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:20.685330 22259 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:44:20.685482 22259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal
uuid: "f77c30f7f2554020b85fa930af9287c5"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a05feb899ca981c28b4f0d5b28a6ae79"
server_key_iv: "69fbf6b749bf8ffa11bf5c3e00f121b7"
server_key_version: "encryptionkey@0"
I20260810 06:44:20.685667 22259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:20.698793 22259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:20.702245 22259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:20.702479 22259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:20.718884 22259 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.60:41639
I20260810 06:44:20.718914 22520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.60:41639 every 8 connection(s)
I20260810 06:44:20.720103 22259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
I20260810 06:44:20.723515 22521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:20.730020 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 22259
I20260810 06:44:20.729753 22521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344260, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.60@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.60@KRBTEST.COM
I20260810 06:44:20.750337 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.732687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.10.242.62:34471)
0810 06:44:20.732849 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.732855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.733593 (+ 738us) server_negotiation.cc:407] Connection header received
0810 06:44:20.735186 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.735192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.735271 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.735369 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.736962 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.738063 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.740090 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.740575 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.740700 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.744213 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.744218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.744222 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.744256 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.746939 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.747870 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.747873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.747876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.748011 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.748461 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.748464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.748466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.748760 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.748884 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.749774 (+ 890us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.749961 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 06:44:20.751217 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.732627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:48444)
0810 06:44:20.733440 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.733481 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.734921 (+ 1440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.735449 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.735456 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.736284 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.736789 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.736813 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.738239 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.738245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.739957 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.739971 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.740794 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.740799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.740956 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.741590 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.741617 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.744038 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.747087 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.747098 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.747110 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.747762 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.748113 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.748118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.748122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.748354 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.748873 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.748880 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.749281 (+ 401us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.749572 (+ 291us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.750040 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":433,"threads_started":1}
I20260810 06:44:20.751782 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.732109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57000 (local address 127.10.242.60:41639)
0810 06:44:20.733096 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.733109 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.733147 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:44:20.734920 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.734940 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.735191 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.735402 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.737117 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.738621 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.740365 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.741040 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.741128 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.745500 (+ 4372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.745508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.745532 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.745560 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.748641 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.749402 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.749409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.749411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.749539 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.750035 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.750041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.750044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.750502 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.750806 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.751134 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.751362 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":678,"threads_started":1}
I20260810 06:44:20.752828 22521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344260, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.60@KRBTEST.COM for oryx/127.10.242.61@KRBTEST.COM
I20260810 06:44:20.764855 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.753417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.0.0.1:51590)
0810 06:44:20.753585 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.753602 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.753682 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.754402 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.754406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.754778 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.754898 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.754906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.756126 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.756132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.757053 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.757072 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.757696 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.757701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.757893 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.758375 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.758390 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.760136 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.762689 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.762694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.762698 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.763045 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.763395 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.763400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.763404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.763528 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.764018 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.764023 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.764165 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.764280 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.764538 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:44:20.764938 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.753591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.10.242.61:44177)
0810 06:44:20.754080 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.754084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.754101 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:44:20.754183 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.754187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.754263 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.754366 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.755037 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.755994 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.757189 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.757567 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.757673 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.760316 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.760321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.760323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.760354 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.762576 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.763151 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.763155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.763157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.763295 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.763625 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.763629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.763631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.763889 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.764026 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.764386 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.764591 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
I20260810 06:44:20.765793 22521 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344260, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.60@KRBTEST.COM for oryx/127.10.242.60@KRBTEST.COM
I20260810 06:44:20.776363 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.766462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.0.0.1:57008)
0810 06:44:20.766650 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.766668 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.766750 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.766964 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.766967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.767270 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.767395 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.767403 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.768236 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.768239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.769046 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.769056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.769629 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.769633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.769791 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.770223 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.770239 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.771975 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.774569 (+ 2594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.774572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.774574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.774884 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.775156 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.775159 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.775161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.775304 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.775720 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.775723 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.775835 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.775932 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.776091 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:44:20.776418 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.766622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57008 (local address 127.10.242.60:41639)
0810 06:44:20.766748 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.766752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.766767 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:44:20.766822 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.766825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.766872 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.766939 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.767499 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.768134 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.769156 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.769517 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.769606 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.772137 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.772142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.772145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.772172 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.774465 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.774971 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.774975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.774977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.775079 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.775384 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.775387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.775389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.775632 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.775731 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.776015 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.776205 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":46}
I20260810 06:44:20.780725 22521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: Bootstrap starting.
I20260810 06:44:20.783768 22521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:20.784756 22521 log.cc:824] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:20.787178 22521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: No bootstrap required, opened a new log
I20260810 06:44:20.794049 22521 raft_consensus.cc:348] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } }
I20260810 06:44:20.794409 22521 raft_consensus.cc:374] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:20.794514 22521 raft_consensus.cc:729] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77c30f7f2554020b85fa930af9287c5, State: Initialized, Role: FOLLOWER
I20260810 06:44:20.795030 22521 consensus_queue.cc:260] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } }
I20260810 06:44:20.795822 22533 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } } }
I20260810 06:44:20.796113 22533 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:44:20.796718 22521 sys_catalog.cc:564] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:44:20.805092 22548 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:44:20.805233 22548 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:44:20.806034 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 21434
I20260810 06:44:20.810287 22259 master_runner.cc:428] Detected that this master f77c30f7f2554020b85fa930af9287c5 is joining an existing cluster
I20260810 06:44:20.810389 22259 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.242.60:41639
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.1@KRBTEST.COM
I20260810 06:44:20.828707 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.807023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.10.242.1:45429)
0810 06:44:20.808087 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.808096 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.808124 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:44:20.808209 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.808218 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.808436 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.808595 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.810005 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.811583 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.813396 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.814269 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.814404 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.820495 (+ 6091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.820512 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.820530 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.820578 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.824554 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.825480 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.825491 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.825495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.825694 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.826242 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.826248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.826252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.826754 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.827084 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.827843 (+ 759us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.828166 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":887,"thread_start_us":756,"threads_started":1}
I20260810 06:44:20.831284 21633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:20.846187 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.815040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51598 (local address 127.10.242.61:44177)
0810 06:44:20.815207 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.815213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.830680 (+ 15467us) server_negotiation.cc:407] Connection header received
0810 06:44:20.830969 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.830974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.831084 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.831157 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.832908 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.834043 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.836233 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.836921 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.837015 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.839677 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.839684 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.839689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.839730 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.843004 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.843766 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.843772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.843776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.843924 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.844393 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.844398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.844402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.844776 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.844910 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.845542 (+ 632us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.845795 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:44:20.847363 22564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.815055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.0.0.1:51598)
0810 06:44:20.830556 (+ 15501us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.830576 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.830807 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.831383 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.831388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.832208 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.832695 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.832710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.834261 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.834267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.836059 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.836076 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.837316 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.837322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.837488 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.838193 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.838217 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.839447 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.843155 (+ 3708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.843160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.843165 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.843630 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.844064 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.844068 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.844071 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.844253 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.844996 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.845001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.845218 (+ 217us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.846475 (+ 1257us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.846868 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15308,"mutex_wait_us":1,"thread_start_us":15030,"threads_started":1}
I20260810 06:44:20.850478 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.814859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48450 (local address 127.10.242.62:34471)
0810 06:44:20.815219 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.815224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.831074 (+ 15850us) server_negotiation.cc:407] Connection header received
0810 06:44:20.831362 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.831368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.831451 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.831523 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.833066 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.834275 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.836879 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.837526 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.837622 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.841779 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.841786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.841790 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.841830 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.845313 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.846375 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.846380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.846384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.846539 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.847104 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.847110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.847114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.847483 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.847615 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.849860 (+ 2245us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.850094 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260810 06:44:20.853470 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.815463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:48450)
0810 06:44:20.830967 (+ 15504us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.830985 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.831195 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.831737 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.831741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.832531 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.832884 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.832896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.834655 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.834660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.836702 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.836717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.837903 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.837910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.838268 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.838874 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.838896 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.840156 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.845536 (+ 5380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.845541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.845544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.846039 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.846741 (+ 702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.846745 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.846749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.846930 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.849277 (+ 2347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.849281 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.849547 (+ 266us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.850740 (+ 1193us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.852045 (+ 1305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15361,"mutex_wait_us":1296,"thread_start_us":21599,"threads_started":1}
I20260810 06:44:20.859366 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.852004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.0.0.1:51602)
0810 06:44:20.852883 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.852900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.853019 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.853668 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.853675 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.853705 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:20.853924 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.853936 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.855315 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.855320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.857208 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.857220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.858684 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.858688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.858819 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.858855 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.858938 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.859060 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":623,"threads_started":1}
I20260810 06:44:20.859366 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.852363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51602 (local address 127.10.242.61:44177)
0810 06:44:20.852527 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.852532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.852995 (+ 463us) server_negotiation.cc:407] Connection header received
0810 06:44:20.853127 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.853132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.853225 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.853341 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:20.854088 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.855141 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.857356 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.858204 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.858336 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.858557 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.858984 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.859138 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:44:20.860484 22335 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 06:44:20.861039 22377 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6445ff13d22498e9dc52884364e67c0. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } } }
I20260810 06:44:20.861281 22377 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:44:20.875048 22564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.838332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.0.0.1:57010)
0810 06:44:20.847690 (+ 9358us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.847751 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.847895 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.848305 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.848309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.848864 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.849081 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.849093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.850635 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.850640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.851929 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.851944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.853113 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.853118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.856847 (+ 3729us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.866357 (+ 9510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.866385 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.867871 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.872237 (+ 4366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.872242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.872246 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.872760 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.873174 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.873179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.873183 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.873344 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.874030 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.874035 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.874215 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.874363 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.874659 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9242,"thread_start_us":30233,"threads_started":1}
I20260810 06:44:20.876039 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.837672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57010 (local address 127.10.242.60:41639)
0810 06:44:20.837850 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.837856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.847815 (+ 9959us) server_negotiation.cc:407] Connection header received
0810 06:44:20.848010 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.848015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.848075 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.848130 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.849234 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.850390 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.852073 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.852907 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.853003 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.868046 (+ 15043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.868051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.868056 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.868088 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.872083 (+ 3995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.872887 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.872892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.872895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.873047 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.873454 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.873458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.873462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.873830 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.873975 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.875479 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.875706 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 06:44:20 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.62@KRBTEST.COM for oryx/127.10.242.60@KRBTEST.COM
I20260810 06:44:20.894835 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.873895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.0.0.1:57024)
0810 06:44:20.878136 (+ 4241us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.878169 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.878325 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.878917 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.878922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.879380 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.879556 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.879568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.881895 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.881900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.883276 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.883291 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.884785 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.884791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.884967 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.885572 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:20.885596 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:20.888251 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:20.892268 (+ 4017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.892273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.892277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.892785 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.893149 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:20.893154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:20.893158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:20.893316 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:20.893904 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:20.893909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:20.894050 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 06:44:20.894228 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 06:44:20.894480 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4129}
I20260810 06:44:20.894922 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.874211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57024 (local address 127.10.242.60:41639)
0810 06:44:20.878000 (+ 3789us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.878006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.878228 (+ 222us) server_negotiation.cc:407] Connection header received
0810 06:44:20.878504 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.878510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.878599 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.878677 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.879749 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.881721 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.883730 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.884424 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.884522 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:20.888450 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:20.888457 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:20.888462 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:20.888494 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:20.892117 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.892907 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.892912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.892916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.893043 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:20.893431 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:20.893437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:20.893440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:20.893788 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:20.893930 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:20.894335 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 06:44:20.894544 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3687,"thread_start_us":16120,"threads_started":1}
I20260810 06:44:20.896646 21335 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.242.60:41639: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f77c30f7f2554020b85fa930af9287c5"
member_type: NON_VOTER
last_known_addr {
host: "127.10.242.60"
port: 41639
}
attrs {
promote: true
}
}
dest_uuid: "f6445ff13d22498e9dc52884364e67c0"
cas_config_opid_index: 5
I20260810 06:44:20.897660 21335 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } }
I20260810 06:44:20.899647 22573 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f77c30f7f2554020b85fa930af9287c5 (127.10.242.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:20.902858 22335 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 LEARNER]: Refusing update from remote peer f6445ff13d22498e9dc52884364e67c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 06:44:20.904845 22573 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.242.60:41639
I20260810 06:44:20.905368 22259 master.cc:537] Master@127.10.242.60:41639 shutting down...
I20260810 06:44:20.916769 22259 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:20.917215 22259 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:20.917480 22259 tablet_replica.cc:330] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: stopping tablet replica
I20260810 06:44:20.926980 22578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 06:44:20.929080 22259 master.cc:552] Master@127.10.242.60:41639 shutdown complete.
I20260810 06:44:20.929246 22259 master_runner.cc:305] Clearing existing system tablet
I20260810 06:44:20.929649 22259 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:20.929823 22259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:20.927620 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.927049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.0.0.1:57028)
0810 06:44:20.927320 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.927425 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260810 06:44:20.937579 21319 consensus_peers.cc:487] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 -> Peer f77c30f7f2554020b85fa930af9287c5 (127.10.242.60:41639): Couldn't send request to peer f77c30f7f2554020b85fa930af9287c5. Status: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:44:20.928043 22573 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:20.938026 22573 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:20.948401 22335 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f77c30f7f2554020b85fa930af9287c5 (127.10.242.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:20.951105 22259 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.006s sys 0.005s
I20260810 06:44:20.952777 22377 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: NON_VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: true } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:20.953035 22377 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:44:20.953958 22573 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: attempting to promote NON_VOTER ab293efd178e4e7698d549af49df8d67 to VOTER
I20260810 06:44:20.954767 22573 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } }
I20260810 06:44:20.956780 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.956393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.0.0.1:57030)
0810 06:44:20.956549 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.956634 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 06:44:20.957209 22334 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 LEARNER]: Refusing update from remote peer f6445ff13d22498e9dc52884364e67c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 06:44:20.957798 22573 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 06:44:20.958000 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.957554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.0.0.1:57044)
0810 06:44:20.957749 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.957836 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260810 06:44:20.958303 21319 consensus_peers.cc:487] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 -> Peer f77c30f7f2554020b85fa930af9287c5 (127.10.242.60:41639): Couldn't send request to peer f77c30f7f2554020b85fa930af9287c5. Status: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:44:20.960474 22573 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ab293efd178e4e7698d549af49df8d67 (127.10.242.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:20.965246 22334 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ab293efd178e4e7698d549af49df8d67 (127.10.242.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:20.968986 22377 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:20.969239 22377 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:44:20.974517 22573 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:20.974784 22573 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:20.975055 22578 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:20.975344 22578 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:20.976331 22585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:20.976692 22259 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:20.976976 22259 fs_manager.cc:725] Time spent opening block manager: real 0.025s user 0.017s sys 0.008s
I20260810 06:44:20.977084 22259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal
uuid: "f77c30f7f2554020b85fa930af9287c5"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a05feb899ca981c28b4f0d5b28a6ae79"
server_key_iv: "69fbf6b749bf8ffa11bf5c3e00f121b7"
server_key_version: "encryptionkey@0"
I20260810 06:44:20.982234 22259 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:20.989061 22259 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:44:20.989223 22259 log.cc:1196] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 06:44:20.989689 22259 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: Deleting consensus metadata
I20260810 06:44:20.989964 22259 master_runner.cc:315] Copying system tablet from 127.10.242.62:34471
I20260810 06:44:20.992717 22259 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: tablet copy: Beginning tablet copy session from remote peer at address 127.10.242.62:34471
I20260810 06:44:21.008508 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.993513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:48464)
0810 06:44:20.994124 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:20.994143 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:20.994306 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:20.994663 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:20.994668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:20.995186 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:20.995462 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.995475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.996885 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.996891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.998704 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:20.998720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.999673 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.999680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:20.999877 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.000606 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:21.000641 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:21.001757 (+ 1116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:21.005583 (+ 3826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:21.005588 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:21.005593 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:21.006125 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:21.006544 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:21.006549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:21.006553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:21.006731 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:21.007380 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:21.007386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:21.007578 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 06:44:21.007732 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 06:44:21.008045 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":352,"threads_started":1}
I20260810 06:44:21.008515 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:20.993496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.10.242.62:34471)
0810 06:44:20.993758 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:20.993764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:20.994241 (+ 477us) server_negotiation.cc:407] Connection header received
0810 06:44:20.994429 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:20.994434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:20.994513 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:20.994606 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:20.995619 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.996748 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.998846 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:20.999464 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:20.999658 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.001977 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:21.001983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:21.001988 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:21.002025 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:21.005450 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.006274 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.006279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.006283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.006432 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.006833 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.006838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.006842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.007249 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:21.007398 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:21.007885 (+ 487us) server_negotiation.cc:299] Negotiation successful
0810 06:44:21.008132 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 06:44:21.009524 21369 tablet_copy_service.cc:140] P f6445ff13d22498e9dc52884364e67c0: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f77c30f7f2554020b85fa930af9287c5 ({username='oryx', principal='oryx/127.10.242.60@KRBTEST.COM'} at 127.0.0.1:48464)
I20260810 06:44:21.009691 21369 tablet_copy_service.cc:161] P f6445ff13d22498e9dc52884364e67c0: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f77c30f7f2554020b85fa930af9287c5 at {username='oryx', principal='oryx/127.10.242.60@KRBTEST.COM'} at 127.0.0.1:48464: session id = f77c30f7f2554020b85fa930af9287c5-00000000000000000000000000000000
I20260810 06:44:21.013593 21369 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:44:21.015465 22259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:44:21.022290 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 21434
I20260810 06:44:21.032629 22259 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: tablet copy: Starting download of 0 data blocks...
I20260810 06:44:21.032984 22259 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: tablet copy: Starting download of 1 WAL segments...
I20260810 06:44:21.035565 22259 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:44:21.040002 21369 tablet_copy_service.cc:342] P f6445ff13d22498e9dc52884364e67c0: Request end of tablet copy session f77c30f7f2554020b85fa930af9287c5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.242.60@KRBTEST.COM'} at 127.0.0.1:48464
I20260810 06:44:21.040139 21369 tablet_copy_service.cc:433] P f6445ff13d22498e9dc52884364e67c0: ending tablet copy session f77c30f7f2554020b85fa930af9287c5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f77c30f7f2554020b85fa930af9287c5
I20260810 06:44:21.051182 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:45429
--local_ip_for_outbound_sockets=127.10.242.1
--tserver_master_addrs=127.10.242.62:34471
--webserver_port=44901
--webserver_interface=127.10.242.1
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab
--principal=oryx/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.242.62:34471,127.10.242.61:44177
--tserver_master_addrs=127.10.242.62:34471,127.10.242.61:44177,127.10.242.60:41639 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:44:21.052461 22259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:21.054906 22615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:21.059787 22616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:21.069662 22622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:21.070401 22259 server_base.cc:1034] running on GCE node
I20260810 06:44:21.070711 22259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:21.071300 22259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:21.072496 22259 hybrid_clock.cc:648] HybridClock initialized: now 1786344261072454 us; error 30 us; skew 500 ppm
I20260810 06:44:21.072713 22259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:21 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344261, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:21.074527 22259 init.cc:377] Logged in from keytab as oryx/127.10.242.60@KRBTEST.COM (short username oryx)
I20260810 06:44:21.075747 22259 webserver.cc:460] Webserver started at http://127.10.242.60:33093/ using document root <none> and password file <none>
I20260810 06:44:21.076032 22259 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:21.076133 22259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:21.078359 22259 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:44:21.079870 22631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:21.080497 22259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:44:21.080605 22259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal
uuid: "f77c30f7f2554020b85fa930af9287c5"
format_stamp: "Formatted at 2026-08-10 06:44:20 on dist-test-slave-zj8q"
server_key: "a05feb899ca981c28b4f0d5b28a6ae79"
server_key_iv: "69fbf6b749bf8ffa11bf5c3e00f121b7"
server_key_version: "encryptionkey@0"
I20260810 06:44:21.080778 22259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:21.091150 22259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:21.094992 22259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:21.120445 22259 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.60:41639
I20260810 06:44:21.120483 22708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.60:41639 every 8 connection(s)
I20260810 06:44:21.121333 22259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
I20260810 06:44:21.127135 22709 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:44:21.131361 22709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: Bootstrap starting.
I20260810 06:44:21.151150 22709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:44:21.151587 22709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: Bootstrap complete.
I20260810 06:44:21.152462 22709 raft_consensus.cc:348] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } }
I20260810 06:44:21.152695 22709 raft_consensus.cc:729] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f77c30f7f2554020b85fa930af9287c5, State: Initialized, Role: LEARNER
I20260810 06:44:21.152971 22709 consensus_queue.cc:260] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } }
I20260810 06:44:21.153843 22714 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:21.154286 22714 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:44:21.154137 22709 sys_catalog.cc:564] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:21.179329 22728 catalog_manager.cc:1259] Loaded cluster ID: 1e4c471737e54d9c8f57671acd4cdbcc
I20260810 06:44:21.179452 22728 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: loading cluster ID for follower catalog manager: success
I20260810 06:44:21.183012 22728 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: acquiring CA information for follower catalog manager: success
I20260810 06:44:21.185405 22728 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:44:21.411837 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:21.405681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57046 (local address 127.10.242.60:41639)
0810 06:44:21.406257 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:21.406262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:21.406280 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:44:21.406345 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:21.406348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:21.406417 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:21.406518 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:21.406948 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.407911 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.410055 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.410908 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.411014 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.411169 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:21.411484 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 06:44:21.411577 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":294,"threads_started":1}
I20260810 06:44:21.411903 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:21.405487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.0.0.1:57046)
0810 06:44:21.405777 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:21.405800 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:21.405961 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:21.406577 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:21.406582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:21.406615 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:21.406815 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:21.406826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.408065 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.408071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:21.409891 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:21.409908 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.411088 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.411095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:21.411328 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.411365 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:44:21.411468 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 06:44:21.411603 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 06:44:21.413003 22671 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 06:44:21.413717 22714 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6445ff13d22498e9dc52884364e67c0. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:21.413992 22714 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: This master's current role is: LEARNER
W20260810 06:44:21.426956 22619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:21.427256 22619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:21.427412 22619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:21.434504 22619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:21.434616 22619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:21.434757 22619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:44:21.444361 22619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab
--principal=oryx/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:45429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=44901
--tserver_master_addrs=127.10.242.62:34471,127.10.242.61:44177,127.10.242.60:41639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:21.445565 22619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:21.446831 22619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:21.459630 22734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:21.459702 22736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:21.459681 22744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:21.460397 22619 server_base.cc:1034] running on GCE node
I20260810 06:44:21.461004 22619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:21.461862 22619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:21.463066 22619 hybrid_clock.cc:648] HybridClock initialized: now 1786344261463021 us; error 32 us; skew 500 ppm
I20260810 06:44:21.463352 22619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:21 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344261, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:21.466514 22619 init.cc:377] Logged in from keytab as oryx/127.10.242.1@KRBTEST.COM (short username oryx)
I20260810 06:44:21.467839 22619 webserver.cc:460] Webserver started at http://127.10.242.1:44901/ using document root <none> and password file <none>
I20260810 06:44:21.468452 22619 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:21.468581 22619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:21.473723 22619 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:44:21.476411 22752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:21.477684 22619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:44:21.477830 22619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "4f53f4935ebb4d38a826be0c54c257df"
format_stamp: "Formatted at 2026-08-10 06:44:18 on dist-test-slave-zj8q"
server_key: "be2e99f1725a1afd2677b7247f2c519c"
server_key_iv: "27b3a6f307225de85fee307be89b1176"
server_key_version: "encryptionkey@0"
I20260810 06:44:21.478812 22619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:21.500674 22619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:21.504679 22619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:21.504930 22619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:21.505946 22619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:21.507326 22619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:21.507419 22619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:21.507521 22619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:21.507592 22619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:21.536478 22619 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:45429
I20260810 06:44:21.536556 22944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:45429 every 8 connection(s)
I20260810 06:44:21.537905 22619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:21.541034 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 22619
I20260810 06:44:21.541381 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 21655
Aug 10 06:44:21 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344261, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.61@KRBTEST.COM
Aug 10 06:44:21 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344261, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
Aug 10 06:44:21 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.2@KRBTEST.COM
Aug 10 06:44:21 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344261, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.60@KRBTEST.COM
I20260810 06:44:21.578560 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:21.541962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:53829 (local address 127.10.242.61:44177)
0810 06:44:21.557047 (+ 15085us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:21.557053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:21.557079 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:44:21.558533 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:21.558539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:21.558633 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:21.558724 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:21.561528 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.562704 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.565708 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.566350 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.566449 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.571267 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:21.571275 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:21.571280 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:21.571323 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:21.574785 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.575680 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.575685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.575689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.575842 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.576405 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.576411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.576414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.576763 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:21.576894 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:21.577889 (+ 995us) server_negotiation.cc:299] Negotiation successful
0810 06:44:21.578148 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14959,"thread_start_us":14844,"threads_started":1}
I20260810 06:44:21.578994 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:21.540626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:33313 (local address 127.10.242.62:34471)
0810 06:44:21.547714 (+ 7088us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:21.547722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:21.554708 (+ 6986us) server_negotiation.cc:407] Connection header received
0810 06:44:21.558462 (+ 3754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:21.558467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:21.558547 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:21.558626 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:21.561536 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.562692 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.565698 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.566318 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.566411 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.571979 (+ 5568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:21.571986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:21.571991 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:21.572030 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:21.575939 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.576749 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.576754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.576758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.576894 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.577355 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.577360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.577362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.577741 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:21.577865 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:21.578414 (+ 549us) server_negotiation.cc:299] Negotiation successful
0810 06:44:21.578646 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6930,"thread_start_us":6822,"threads_started":1}
I20260810 06:44:21.579808 22729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:21.555330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:39211 (local address 127.10.242.60:41639)
0810 06:44:21.555530 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:21.555536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:21.556883 (+ 1347us) server_negotiation.cc:407] Connection header received
0810 06:44:21.558506 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:21.558513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:21.558594 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:21.558666 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:21.561529 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.562693 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.565707 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.566393 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.566478 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.573077 (+ 6599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:21.573084 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:21.573088 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:21.573141 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:21.576443 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.577231 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.577236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.577240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.577409 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.577842 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.577847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.577850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.578216 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:21.578324 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:21.579040 (+ 716us) server_negotiation.cc:299] Negotiation successful
0810 06:44:21.579485 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 06:44:21.580237 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:21.542401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.10.242.2:44043)
0810 06:44:21.558120 (+ 15719us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:21.558128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:21.558193 (+ 65us) server_negotiation.cc:407] Connection header received
0810 06:44:21.558280 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:21.558289 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:21.558514 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:21.560681 (+ 2167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:21.560916 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.562528 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.564454 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.565430 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.565570 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.572436 (+ 6866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:21.572450 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:21.572467 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:21.572508 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:21.576852 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.577706 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.577715 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.577718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.577877 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:21.578339 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:21.578345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:21.578348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:21.578774 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:21.579041 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:21.579585 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 06:44:21.579833 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15555,"thread_start_us":15395,"threads_started":1}
I20260810 06:44:21.581202 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:21.555688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.10.242.1:39211)
0810 06:44:21.556781 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:21.556798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:21.558119 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:21.559111 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:21.559134 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:21.560187 (+ 1053us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:21.561327 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:21.561354 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.562889 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.562895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:21.565544 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:21.565556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.566780 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.566788 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:21.567285 (+ 497us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.567979 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:21.568030 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:21.572869 (+ 4839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:21.576594 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:21.576599 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:21.576602 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:21.577082 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:21.577551 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:21.577555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:21.577558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:21.577713 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:21.578411 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:21.578415 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:21.578549 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 06:44:21.580157 (+ 1608us) client_negotiation.cc:240] Negotiation successful
0810 06:44:21.580427 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":536,"spinlock_wait_cycles":41344,"thread_start_us":859,"threads_started":1}
I20260810 06:44:21.581207 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:21.541285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.1:33313)
0810 06:44:21.554582 (+ 13297us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:21.554600 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:21.558126 (+ 3526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:21.559111 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:21.559122 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:21.560189 (+ 1067us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:21.561337 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:21.561355 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.562886 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.562891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:21.565511 (+ 2620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:21.565539 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.566773 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.566786 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:21.567179 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.568084 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:21.568115 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:21.571746 (+ 3631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:21.576097 (+ 4351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:21.576103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:21.576106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:21.576594 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:21.577049 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:21.577053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:21.577056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:21.577236 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:21.577957 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:21.577961 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:21.578103 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 06:44:21.579185 (+ 1082us) client_negotiation.cc:240] Negotiation successful
0810 06:44:21.579766 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13148,"mutex_wait_us":739,"spinlock_wait_cycles":84736,"thread_start_us":13281,"threads_started":1}
I20260810 06:44:21.581815 22951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:21.541285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.10.242.1:53829)
0810 06:44:21.552894 (+ 11609us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:21.552948 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:21.558119 (+ 5171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:21.559111 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:21.559122 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:21.560187 (+ 1065us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:21.561327 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:21.561354 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.562889 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.562895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:21.565542 (+ 2647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:21.565556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:21.566803 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:21.566810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:21.567246 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:21.567995 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:21.568026 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:21.571020 (+ 2994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:21.574939 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:21.574949 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:21.574965 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:21.575524 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:21.575980 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:21.575985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:21.575989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:21.576258 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:21.576981 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:21.576990 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:21.577457 (+ 467us) client_negotiation.cc:769] Sending connection context
0810 06:44:21.578868 (+ 1411us) client_negotiation.cc:240] Negotiation successful
0810 06:44:21.581385 (+ 2517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11275,"mutex_wait_us":2451,"spinlock_wait_cycles":2688,"thread_start_us":11040,"threads_started":1}
I20260810 06:44:21.583041 22945 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34471
I20260810 06:44:21.583076 22947 heartbeater.cc:347] Connected to a master server at 127.10.242.61:44177
I20260810 06:44:21.583316 22945 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:21.583323 22947 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:21.584052 21855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:21.584170 22945 heartbeater.cc:511] Master 127.10.242.62:34471 requested a full tablet report, sending...
I20260810 06:44:21.584194 22947 heartbeater.cc:511] Master 127.10.242.61:44177 requested a full tablet report, sending...
I20260810 06:44:21.585341 22946 heartbeater.cc:347] Connected to a master server at 127.10.242.60:41639
I20260810 06:44:21.585464 22946 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:21.585906 22946 heartbeater.cc:511] Master 127.10.242.60:41639 requested a full tablet report, sending...
I20260810 06:44:21.586131 21335 ts_manager.cc:194] Re-registered known tserver with Master: 4f53f4935ebb4d38a826be0c54c257df (127.10.242.1:45429)
I20260810 06:44:21.586699 22316 ts_manager.cc:194] Registered new tserver with Master: 4f53f4935ebb4d38a826be0c54c257df (127.10.242.1:45429)
I20260810 06:44:21.587046 21335 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:33313
I20260810 06:44:21.587638 22655 ts_manager.cc:194] Registered new tserver with Master: 4f53f4935ebb4d38a826be0c54c257df (127.10.242.1:45429)
I20260810 06:44:21.668864 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 21655
I20260810 06:44:21.693384 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:44043
--local_ip_for_outbound_sockets=127.10.242.2
--tserver_master_addrs=127.10.242.62:34471
--webserver_port=36241
--webserver_interface=127.10.242.2
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab
--principal=oryx/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.242.62:34471,127.10.242.61:44177
--tserver_master_addrs=127.10.242.62:34471,127.10.242.61:44177,127.10.242.60:41639 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:44:21.956467 22579 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: NON_VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: true } } }
I20260810 06:44:21.956775 22579 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:21.967015 22964 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: attempting to promote NON_VOTER f77c30f7f2554020b85fa930af9287c5 to VOTER
I20260810 06:44:21.967733 22964 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } }
I20260810 06:44:21.970036 22335 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Refusing update from remote peer f6445ff13d22498e9dc52884364e67c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 06:44:21.970114 22671 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 LEARNER]: Refusing update from remote peer f6445ff13d22498e9dc52884364e67c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 06:44:21.970721 22579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 06:44:21.971123 22964 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 06:44:21.973590 22579 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f77c30f7f2554020b85fa930af9287c5 (127.10.242.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:21.975018 22335 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f77c30f7f2554020b85fa930af9287c5 (127.10.242.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:21.975261 22671 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f77c30f7f2554020b85fa930af9287c5 (127.10.242.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:21.977404 22579 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:21.977627 22579 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:21.978251 22966 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:21.978520 22966 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:21.979285 22966 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:21.979499 22966 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:21.983001 22973 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:21.983444 22973 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:44:21.983198 22972 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6445ff13d22498e9dc52884364e67c0" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:21.983675 22972 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:44:22.029309 22962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:22.029575 22962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:22.029708 22962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:22.036736 22962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:22.036849 22962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:22.036995 22962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:22.046533 22962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab
--principal=oryx/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:44043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=36241
--tserver_master_addrs=127.10.242.62:34471,127.10.242.61:44177,127.10.242.60:41639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:22.047909 22962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:22.049284 22962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:22.062389 22980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:22.062409 22979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:22.063006 22982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:22.063112 22962 server_base.cc:1034] running on GCE node
I20260810 06:44:22.063796 22962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:22.064829 22962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:22.066044 22962 hybrid_clock.cc:648] HybridClock initialized: now 1786344262065983 us; error 36 us; skew 500 ppm
I20260810 06:44:22.066505 22962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344262, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:22.070252 22962 init.cc:377] Logged in from keytab as oryx/127.10.242.2@KRBTEST.COM (short username oryx)
I20260810 06:44:22.071967 22962 webserver.cc:460] Webserver started at http://127.10.242.2:36241/ using document root <none> and password file <none>
I20260810 06:44:22.072760 22962 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:22.072904 22962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:22.078831 22962 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:44:22.081768 22993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:22.083361 22962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:22.083515 22962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "c8fb35a3e0384cdbad640fe4524933a8"
format_stamp: "Formatted at 2026-08-10 06:44:19 on dist-test-slave-zj8q"
server_key: "58f1ee180636ee9af9f633981409e2b1"
server_key_iv: "9f5fbb28a9cbac5797eecad2084eb984"
server_key_version: "encryptionkey@0"
I20260810 06:44:22.084488 22962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:22.096017 22962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:22.100103 22962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:22.100371 22962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:22.101279 22962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:22.102644 22962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:22.102747 22962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:22.102850 22962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:22.102919 22962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:22.132886 22962 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:44043
I20260810 06:44:22.132916 23165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:44043 every 8 connection(s)
I20260810 06:44:22.134627 22962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:22.136757 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 22962
I20260810 06:44:22.137075 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 21870
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344262, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.60@KRBTEST.COM
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344262, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.61@KRBTEST.COM
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344262, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.3@KRBTEST.COM
I20260810 06:44:22.191314 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.137818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:54865 (local address 127.10.242.60:41639)
0810 06:44:22.166300 (+ 28482us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.166307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.166330 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:44:22.169387 (+ 3057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.169393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.169620 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.169740 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.174115 (+ 4375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.175414 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.178389 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.179038 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.179125 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.184409 (+ 5284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.184417 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.184422 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.184462 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.187859 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.188731 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.188736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.188739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.188885 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.189389 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.189394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.189398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.189666 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.189806 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.190725 (+ 919us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.190936 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28284,"thread_start_us":28546,"threads_started":1}
I20260810 06:44:22.191906 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.138500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:52013 (local address 127.10.242.61:44177)
0810 06:44:22.165660 (+ 27160us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.165668 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.170292 (+ 4624us) server_negotiation.cc:407] Connection header received
0810 06:44:22.170358 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.170363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.170458 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.170560 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.174112 (+ 3552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.175401 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.178884 (+ 3483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.179446 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.179559 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.184716 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.184722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.184726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.184780 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.188517 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.189365 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.189371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.189374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.189522 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.189970 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.189975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.189978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.190383 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.190538 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.191341 (+ 803us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.191559 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27034,"thread_start_us":26913,"threads_started":1}
I20260810 06:44:22.193507 23178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.138143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.10.242.2:52013)
0810 06:44:22.170012 (+ 31869us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:22.170030 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:22.170176 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:22.170745 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:22.170749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:22.172702 (+ 1953us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:22.173869 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.173899 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.175644 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.175650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.178692 (+ 3042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.178709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.179814 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.179821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.179963 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.180687 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:22.180726 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:22.184493 (+ 3767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:22.188664 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.188669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.188672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.189219 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.189661 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.189667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.189671 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.189838 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.190607 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:22.190611 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:22.190766 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 06:44:22.190947 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 06:44:22.192373 (+ 1426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31223,"mutex_wait_us":31,"thread_start_us":30251,"threads_started":1}
I20260810 06:44:22.193833 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.138143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.10.242.2:54865)
0810 06:44:22.163756 (+ 25613us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:22.163832 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:22.167985 (+ 4153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:22.170274 (+ 2289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:22.170286 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:22.171194 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:22.173866 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.173896 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.175649 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.175655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.178206 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.178225 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.179492 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.179500 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.179780 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.180539 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:22.180638 (+ 99us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:22.184149 (+ 3511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:22.188025 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.188035 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.188062 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.188608 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.189043 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.189047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.189050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.189294 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.189809 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:22.189818 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:22.190281 (+ 463us) client_negotiation.cc:769] Sending connection context
0810 06:44:22.191663 (+ 1382us) client_negotiation.cc:240] Negotiation successful
0810 06:44:22.192055 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25167,"mutex_wait_us":66,"thread_start_us":29984,"threads_started":1}
I20260810 06:44:22.195216 23170 heartbeater.cc:347] Connected to a master server at 127.10.242.61:44177
I20260810 06:44:22.195503 23170 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:22.196264 23170 heartbeater.cc:511] Master 127.10.242.61:44177 requested a full tablet report, sending...
I20260810 06:44:22.196293 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.138692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.10.242.3:38667)
0810 06:44:22.163778 (+ 25086us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.163790 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.164746 (+ 956us) server_negotiation.cc:407] Connection header received
0810 06:44:22.165135 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.165146 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.165458 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.165636 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.168317 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.176703 (+ 8386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.179272 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.180530 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.180680 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.188059 (+ 7379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.188073 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.188092 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.188140 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.192401 (+ 4261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.193225 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.193233 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.193236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.193379 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.194139 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.194144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.194147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.194559 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.194840 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.195825 (+ 985us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.196014 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24854,"thread_start_us":25122,"threads_started":1}
I20260810 06:44:22.197032 23169 heartbeater.cc:347] Connected to a master server at 127.10.242.60:41639
I20260810 06:44:22.197165 23169 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:22.197613 23169 heartbeater.cc:511] Master 127.10.242.60:41639 requested a full tablet report, sending...
I20260810 06:44:22.197685 22076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:22.197954 22316 ts_manager.cc:194] Registered new tserver with Master: c8fb35a3e0384cdbad640fe4524933a8 (127.10.242.2:44043)
I20260810 06:44:22.198812 22655 ts_manager.cc:194] Registered new tserver with Master: c8fb35a3e0384cdbad640fe4524933a8 (127.10.242.2:44043)
I20260810 06:44:22.199978 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.169564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:35353)
0810 06:44:22.177219 (+ 7655us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:22.177237 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:22.177350 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:22.177960 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:22.177965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:22.178626 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:22.178960 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.178972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.180357 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.180362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.181965 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.181979 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.183083 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.183090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.183283 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.184414 (+ 1131us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:22.184436 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:22.187299 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:22.192566 (+ 5267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.192571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.192575 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.193078 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.193575 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.193580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.193584 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.193762 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.194960 (+ 1198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:22.194965 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:22.195127 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 06:44:22.198927 (+ 3800us) client_negotiation.cc:240] Negotiation successful
0810 06:44:22.199526 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7533,"spinlock_wait_cycles":28416,"thread_start_us":7392,"threads_started":1}
I20260810 06:44:22.201030 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.169743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:35353 (local address 127.10.242.62:34471)
0810 06:44:22.177507 (+ 7764us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.177513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.177531 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:44:22.177586 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.177592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.177684 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.177783 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.179185 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.180219 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.182208 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.182877 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.183017 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.187518 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.187524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.187529 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.187649 (+ 120us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.192420 (+ 4771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.193209 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.193214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.193217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.193379 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.193913 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.193919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.193923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.194748 (+ 825us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.194903 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.200350 (+ 5447us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.200607 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7635,"thread_start_us":7494,"threads_started":1}
I20260810 06:44:22.201782 23167 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34471
I20260810 06:44:22.201902 23167 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:22.202317 23167 heartbeater.cc:511] Master 127.10.242.62:34471 requested a full tablet report, sending...
I20260810 06:44:22.203156 21335 ts_manager.cc:194] Re-registered known tserver with Master: c8fb35a3e0384cdbad640fe4524933a8 (127.10.242.2:44043)
I20260810 06:44:22.203966 21335 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:35353
I20260810 06:44:22.281159 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 21870
I20260810 06:44:22.307029 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:38667
--local_ip_for_outbound_sockets=127.10.242.3
--tserver_master_addrs=127.10.242.62:34471
--webserver_port=32985
--webserver_interface=127.10.242.3
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab
--principal=oryx/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:34775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.242.62:34471,127.10.242.61:44177
--tserver_master_addrs=127.10.242.62:34471,127.10.242.61:44177,127.10.242.60:41639 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:44:22.589332 22945 heartbeater.cc:503] Master 127.10.242.62:34471 was elected leader, sending a full tablet report...
W20260810 06:44:22.633904 23187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:22.634258 23187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:22.634375 23187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:22.641315 23187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:22.641414 23187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:22.641522 23187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:22.651124 23187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:34775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab
--principal=oryx/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:38667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=32985
--tserver_master_addrs=127.10.242.62:34471,127.10.242.61:44177,127.10.242.60:41639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:22.652328 23187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:22.653612 23187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:22.666529 23195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:22.666469 23196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:22.666556 23199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:22.667371 23187 server_base.cc:1034] running on GCE node
I20260810 06:44:22.667973 23187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:22.668869 23187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:22.670185 23187 hybrid_clock.cc:648] HybridClock initialized: now 1786344262670101 us; error 46 us; skew 500 ppm
I20260810 06:44:22.670492 23187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344262, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:22.673447 23187 init.cc:377] Logged in from keytab as oryx/127.10.242.3@KRBTEST.COM (short username oryx)
I20260810 06:44:22.674857 23187 webserver.cc:460] Webserver started at http://127.10.242.3:32985/ using document root <none> and password file <none>
I20260810 06:44:22.675613 23187 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:22.675746 23187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:22.680908 23187 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:44:22.683579 23210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:22.684820 23187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:44:22.684971 23187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "a8ab00aad90243d29326e71b495f4638"
format_stamp: "Formatted at 2026-08-10 06:44:19 on dist-test-slave-zj8q"
server_key: "a40f3c822ca585b31995e22c837de5f0"
server_key_iv: "cfadefdb28956319f798d3080484f8e7"
server_key_version: "encryptionkey@0"
I20260810 06:44:22.685954 23187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:22.698954 23187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:22.702821 23187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:22.703065 23187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:22.703933 23187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:22.705237 23187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:22.705334 23187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:22.705436 23187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:22.705529 23187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:22.734797 23187 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:38667
I20260810 06:44:22.734861 23408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:38667 every 8 connection(s)
I20260810 06:44:22.736218 23187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:22.738627 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 23187
I20260810 06:44:22.781337 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.748665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46214 (local address 127.10.242.62:34471)
0810 06:44:22.757612 (+ 8947us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.757620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.762894 (+ 5274us) server_negotiation.cc:407] Connection header received
0810 06:44:22.762976 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.762983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.763093 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.763202 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.766784 (+ 3582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.768063 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.770383 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.771089 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.771203 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.776049 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.776054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.776058 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.776093 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.778751 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.779420 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.779424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.779426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.779576 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.779929 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.779933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.779935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.780191 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.780412 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.780775 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.780991 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8785,"thread_start_us":8666,"threads_started":1}
I20260810 06:44:22.786670 21335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:44:22.797878 21335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:44:22.798506 23417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.762915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.10.242.60:41639)
0810 06:44:22.781063 (+ 18148us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.781068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.781088 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:44:22.781151 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.781157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.781235 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.781373 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.783441 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.784712 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.786840 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.787446 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.787578 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.790987 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.790995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.791000 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.791048 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.794399 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.795260 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.795266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.795270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.795420 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.795886 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.795891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.795893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.796259 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.796484 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.797571 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.797797 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18042,"thread_start_us":15218,"threads_started":1}
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344262, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.61@KRBTEST.COM
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344262, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.60@KRBTEST.COM
I20260810 06:44:22.825152 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.748920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.10.242.61:44177)
0810 06:44:22.783847 (+ 34927us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.783853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.783871 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:44:22.783933 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.783938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.784023 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.789588 (+ 5565us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.789776 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.794561 (+ 4785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.796830 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.797533 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.798328 (+ 795us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.801263 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.801270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.801276 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.801312 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.815240 (+ 13928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.817933 (+ 2693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.817938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.817941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.818115 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.818911 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.818916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.818919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.819296 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.819535 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.824503 (+ 4968us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.824726 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32880,"thread_start_us":11316,"threads_started":1}
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344262, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:22.831849 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.739174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:51775 (local address 127.10.242.61:44177)
0810 06:44:22.760235 (+ 21061us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.760242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.782922 (+ 22680us) server_negotiation.cc:407] Connection header received
0810 06:44:22.791259 (+ 8337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.791266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.791347 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.791423 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.802690 (+ 11267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.804004 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.813572 (+ 9568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.814381 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.814527 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.821953 (+ 7426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.821961 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.821966 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.822017 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.825642 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.827418 (+ 1776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.827424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.827427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.827614 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.828177 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.828183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.828186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.828550 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.828672 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.831240 (+ 2568us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.831461 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20941,"thread_start_us":55234,"threads_started":1}
I20260810 06:44:22.834071 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.739613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.10.242.3:51775)
0810 06:44:22.782741 (+ 43128us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:22.782808 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:22.791080 (+ 8272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:22.792207 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:22.792222 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:22.793244 (+ 1022us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:22.794245 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.794273 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.805552 (+ 11279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.805557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.807912 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.807931 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.814941 (+ 7010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.814950 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.815266 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.816743 (+ 1477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:22.816782 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:22.821655 (+ 4873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:22.826051 (+ 4396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.826060 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.826080 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.826723 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.827768 (+ 1045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.827772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.827775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.828018 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.828769 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:22.828777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:22.829283 (+ 506us) client_negotiation.cc:769] Sending connection context
0810 06:44:22.832034 (+ 2751us) client_negotiation.cc:240] Negotiation successful
0810 06:44:22.832597 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42772,"thread_start_us":43973,"threads_started":1}
I20260810 06:44:22.837677 23414 heartbeater.cc:347] Connected to a master server at 127.10.242.61:44177
I20260810 06:44:22.838023 23414 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:22.839092 23414 heartbeater.cc:511] Master 127.10.242.61:44177 requested a full tablet report, sending...
I20260810 06:44:22.839696 23419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.740112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.10.242.3:36887)
0810 06:44:22.783397 (+ 43285us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:22.783419 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:22.786937 (+ 3518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:22.804419 (+ 17482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:22.804424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:22.804994 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:22.805305 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.805318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.811237 (+ 5919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.811243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.812909 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.812926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.818431 (+ 5505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.818436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.818604 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.819680 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:22.819701 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:22.823360 (+ 3659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:22.836027 (+ 12667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.836032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.836035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.836542 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.836997 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.837002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.837005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.837736 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.838518 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:22.838523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:22.838704 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 06:44:22.838856 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 06:44:22.839213 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43171,"thread_start_us":42170,"threads_started":1}
I20260810 06:44:22.840668 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.739441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:36887 (local address 127.10.242.60:41639)
0810 06:44:22.778111 (+ 38670us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.778118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.783737 (+ 5619us) server_negotiation.cc:407] Connection header received
0810 06:44:22.802262 (+ 18525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.802269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.802350 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.802431 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.809643 (+ 7212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.811009 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.815696 (+ 4687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.816341 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.816436 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.823597 (+ 7161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.823605 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.823609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.823650 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.835873 (+ 12223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.836680 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.836685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.836689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.836854 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.837877 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.837882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.837885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.838303 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.838433 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.840025 (+ 1592us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.840275 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38527,"thread_start_us":42050,"threads_started":1}
I20260810 06:44:22.841831 23411 heartbeater.cc:347] Connected to a master server at 127.10.242.60:41639
I20260810 06:44:22.841946 23411 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:22.842429 23411 heartbeater.cc:511] Master 127.10.242.60:41639 requested a full tablet report, sending...
I20260810 06:44:22.843008 22316 ts_manager.cc:194] Registered new tserver with Master: a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667)
I20260810 06:44:22.844254 22655 ts_manager.cc:194] Registered new tserver with Master: a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667)
I20260810 06:44:22.844720 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.783648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:50567 (local address 127.10.242.62:34471)
0810 06:44:22.783810 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.783815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.815501 (+ 31686us) server_negotiation.cc:407] Connection header received
0810 06:44:22.815944 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.815950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.816034 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.816137 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.817876 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.819092 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.825880 (+ 6788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.826616 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.826737 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.835618 (+ 8881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.835624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.835629 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.835666 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.839436 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.841557 (+ 2121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.841562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.841566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.841725 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.842883 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.842887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.842890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.843275 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.843403 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.844174 (+ 771us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.844406 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:44:22.845549 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.783558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.3:50567)
0810 06:44:22.815403 (+ 31845us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:22.815424 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:22.815555 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:22.816743 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:22.816748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:22.817355 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:22.817726 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.817739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.821504 (+ 3765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.821509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.825745 (+ 4236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.825760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.826860 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.826867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.827052 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.829720 (+ 2668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:22.829743 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:22.835278 (+ 5535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:22.840927 (+ 5649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.840932 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.840936 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.841415 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.842531 (+ 1116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.842536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.842540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.842734 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.843480 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:22.843485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:22.843859 (+ 374us) client_negotiation.cc:769] Sending connection context
0810 06:44:22.844874 (+ 1015us) client_negotiation.cc:240] Negotiation successful
0810 06:44:22.845169 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29747,"thread_start_us":29606,"threads_started":1}
I20260810 06:44:22.846395 23409 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34471
I20260810 06:44:22.846527 23409 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:22.846987 23409 heartbeater.cc:511] Master 127.10.242.62:34471 requested a full tablet report, sending...
I20260810 06:44:22.847873 21335 ts_manager.cc:194] Re-registered known tserver with Master: a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667)
I20260810 06:44:22.849215 21335 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:50567
I20260810 06:44:22.871340 23456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.847062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.10.242.1:45429)
0810 06:44:22.849740 (+ 2678us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.849749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.863048 (+ 13299us) server_negotiation.cc:407] Connection header received
0810 06:44:22.863178 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.863188 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.863468 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.863684 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:22.864963 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.866518 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.869096 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.870266 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.870403 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.870614 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.870878 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.871036 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":406,"threads_started":1}
I20260810 06:44:22.871403 23454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.846905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45429 (local address 127.0.0.1:44620)
0810 06:44:22.862736 (+ 15831us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:22.862757 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:22.862904 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:22.864233 (+ 1329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:22.864238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:22.864264 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:22.864763 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.864776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.866845 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.866850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.868914 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.868930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.870488 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.870496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.870685 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.870721 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:44:22.870817 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:44:22.870947 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15695,"spinlock_wait_cycles":172800,"thread_start_us":15544,"threads_started":1}
I20260810 06:44:22.874221 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.847923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:44043 (local address 127.0.0.1:50604)
0810 06:44:22.863519 (+ 15596us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:22.863536 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:22.863650 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:22.867205 (+ 3555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:22.867210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:22.867238 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:22.867688 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.867702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.869590 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.869596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.871641 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.871653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.873450 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.873457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.873631 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.873665 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:44:22.873756 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:44:22.873882 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15478,"thread_start_us":15942,"threads_started":1}
Aug 10 06:44:22 dist-test-slave-zj8q krb5kdc[21270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344257, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.62@KRBTEST.COM for oryx/127.10.242.3@KRBTEST.COM
I20260810 06:44:22.874651 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.848162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50604 (local address 127.10.242.2:44043)
0810 06:44:22.866638 (+ 18476us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.866647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.866673 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:44:22.866763 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.866770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.866988 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.867194 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:22.867910 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.869450 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.871782 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.872952 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.873088 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.873281 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.874174 (+ 893us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.874346 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18303,"thread_start_us":18205,"threads_started":1}
I20260810 06:44:22.875655 22826 tablet_service.cc:1467] Processing CreateTablet for tablet b501738f1d4e4d7fb3df9e7b901bc5dc (DEFAULT_TABLE table=test-table [id=23844f2d21e6440ba777d444b24d6e26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:22.877136 22826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b501738f1d4e4d7fb3df9e7b901bc5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:22.877632 23068 tablet_service.cc:1467] Processing CreateTablet for tablet b501738f1d4e4d7fb3df9e7b901bc5dc (DEFAULT_TABLE table=test-table [id=23844f2d21e6440ba777d444b24d6e26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:22.878748 23068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b501738f1d4e4d7fb3df9e7b901bc5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:22.883114 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.848839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52012 (local address 127.10.242.3:38667)
0810 06:44:22.863081 (+ 14242us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:22.863089 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:22.864252 (+ 1163us) server_negotiation.cc:407] Connection header received
0810 06:44:22.864541 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:22.864549 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:22.864776 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:22.864895 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:22.866811 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.868247 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.870248 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.870972 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.871082 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.875538 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:22.875551 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:22.875566 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:22.875956 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:22.879973 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.880751 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.880760 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.880763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.880922 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:22.881375 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:22.881380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:22.881383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:22.881833 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:22.882016 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:22.882500 (+ 484us) server_negotiation.cc:299] Negotiation successful
0810 06:44:22.882748 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14076,"thread_start_us":17784,"threads_started":1}
I20260810 06:44:22.883143 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:22.848302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38667 (local address 127.0.0.1:52012)
0810 06:44:22.864143 (+ 15841us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:22.864164 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:22.864357 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:22.865212 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:22.865216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:22.865835 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:22.866119 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.866142 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.868420 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.868429 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.870077 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:22.870095 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:22.871412 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:22.871419 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:22.871748 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:22.872387 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:22.872409 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:22.875324 (+ 2915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:22.880114 (+ 4790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.880119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.880123 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.880589 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.881043 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:22.881047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:22.881051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:22.881233 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:22.881969 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:22.881975 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:22.882140 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 06:44:22.882316 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 06:44:22.882651 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15715,"spinlock_wait_cycles":384,"thread_start_us":15575,"threads_started":1}
I20260810 06:44:22.886950 23292 tablet_service.cc:1467] Processing CreateTablet for tablet b501738f1d4e4d7fb3df9e7b901bc5dc (DEFAULT_TABLE table=test-table [id=23844f2d21e6440ba777d444b24d6e26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:22.888269 23292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b501738f1d4e4d7fb3df9e7b901bc5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:22.891403 23461 tablet_bootstrap.cc:492] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: Bootstrap starting.
I20260810 06:44:22.892879 23464 tablet_bootstrap.cc:492] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: Bootstrap starting.
I20260810 06:44:22.895504 23461 tablet_bootstrap.cc:654] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:22.896263 23464 tablet_bootstrap.cc:654] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:22.896853 23461 log.cc:824] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:22.897482 23464 log.cc:824] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:22.899434 23461 tablet_bootstrap.cc:492] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: No bootstrap required, opened a new log
I20260810 06:44:22.899686 23461 ts_tablet_manager.cc:1397] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:44:22.901575 23468 tablet_bootstrap.cc:492] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: Bootstrap starting.
I20260810 06:44:22.903690 23464 tablet_bootstrap.cc:492] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: No bootstrap required, opened a new log
I20260810 06:44:22.903935 23464 ts_tablet_manager.cc:1397] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260810 06:44:22.904861 23468 tablet_bootstrap.cc:654] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:22.906020 23468 log.cc:824] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:22.908867 23468 tablet_bootstrap.cc:492] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: No bootstrap required, opened a new log
I20260810 06:44:22.909143 23468 ts_tablet_manager.cc:1397] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 06:44:22.908937 23461 raft_consensus.cc:348] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } }
I20260810 06:44:22.909345 23461 raft_consensus.cc:374] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:22.909446 23461 raft_consensus.cc:729] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8fb35a3e0384cdbad640fe4524933a8, State: Initialized, Role: FOLLOWER
I20260810 06:44:22.910120 23461 consensus_queue.cc:260] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } }
I20260810 06:44:22.911229 23167 heartbeater.cc:503] Master 127.10.242.62:34471 was elected leader, sending a full tablet report...
I20260810 06:44:22.911829 23461 ts_tablet_manager.cc:1428] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260810 06:44:22.911855 23464 raft_consensus.cc:348] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } }
I20260810 06:44:22.912246 23464 raft_consensus.cc:374] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:22.912338 23464 raft_consensus.cc:729] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f53f4935ebb4d38a826be0c54c257df, State: Initialized, Role: FOLLOWER
I20260810 06:44:22.912954 23464 consensus_queue.cc:260] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } }
I20260810 06:44:22.918663 23468 raft_consensus.cc:348] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } }
I20260810 06:44:22.919082 23468 raft_consensus.cc:374] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:22.919202 23468 raft_consensus.cc:729] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ab00aad90243d29326e71b495f4638, State: Initialized, Role: FOLLOWER
I20260810 06:44:22.919778 23468 consensus_queue.cc:260] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } }
I20260810 06:44:22.920825 23409 heartbeater.cc:503] Master 127.10.242.62:34471 was elected leader, sending a full tablet report...
I20260810 06:44:22.921455 23464 ts_tablet_manager.cc:1428] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: Time spent starting tablet: real 0.017s user 0.010s sys 0.008s
I20260810 06:44:22.923229 23468 ts_tablet_manager.cc:1428] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
W20260810 06:44:22.924266 23171 tablet.cc:2369] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:44:22.994413 23415 tablet.cc:2369] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:44:23.040971 22949 tablet.cc:2369] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:44:23.065181 23472 raft_consensus.cc:482] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:23.065551 23472 raft_consensus.cc:504] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } }
I20260810 06:44:23.067245 23472 leader_election.cc:288] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8fb35a3e0384cdbad640fe4524933a8 (127.10.242.2:44043), a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667)
I20260810 06:44:23.075233 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:23.067573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:38897 (local address 127.10.242.2:44043)
0810 06:44:23.067881 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:23.067887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:23.068247 (+ 360us) server_negotiation.cc:407] Connection header received
0810 06:44:23.068449 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:23.068455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:23.068536 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:23.068626 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:23.069202 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.070316 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.072992 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.074232 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.074389 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:23.074461 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:23.074859 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 06:44:23.074984 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260810 06:44:23.075233 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:23.067845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:56781 (local address 127.10.242.3:38667)
0810 06:44:23.068014 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:23.068019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:23.068413 (+ 394us) server_negotiation.cc:407] Connection header received
0810 06:44:23.068544 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:23.068549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:23.068672 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:23.068785 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:23.069374 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.070457 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.072960 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.074268 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.074399 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:23.074566 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:23.074858 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 06:44:23.074984 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:44:23.075516 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:23.067516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38667 (local address 127.10.242.1:56781)
0810 06:44:23.068320 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:23.068335 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:23.068437 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:23.068822 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:23.068826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:23.068850 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:23.069227 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:23.069238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.070589 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.070597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:23.072827 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:23.072837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.074473 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.074479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:23.074646 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:23.074718 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 06:44:23.074808 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 06:44:23.074998 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":155,"thread_start_us":572,"threads_started":1}
I20260810 06:44:23.075529 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:23.067431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:44043 (local address 127.10.242.1:38897)
0810 06:44:23.068104 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:23.068134 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:23.068310 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:23.068659 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:23.068662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:23.068686 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:23.069063 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:23.069076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.070444 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.070447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:23.072820 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:23.072837 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.074454 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.074461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:23.074708 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:23.074734 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:44:23.074808 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:44:23.074999 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":95,"thread_start_us":325,"threads_started":1}
I20260810 06:44:23.076498 23105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b501738f1d4e4d7fb3df9e7b901bc5dc" candidate_uuid: "4f53f4935ebb4d38a826be0c54c257df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8fb35a3e0384cdbad640fe4524933a8" is_pre_election: true
I20260810 06:44:23.076506 23324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b501738f1d4e4d7fb3df9e7b901bc5dc" candidate_uuid: "4f53f4935ebb4d38a826be0c54c257df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ab00aad90243d29326e71b495f4638" is_pre_election: true
I20260810 06:44:23.077097 23105 raft_consensus.cc:2454] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f53f4935ebb4d38a826be0c54c257df in term 0.
I20260810 06:44:23.077098 23324 raft_consensus.cc:2454] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f53f4935ebb4d38a826be0c54c257df in term 0.
I20260810 06:44:23.077735 22753 leader_election.cc:302] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f53f4935ebb4d38a826be0c54c257df, a8ab00aad90243d29326e71b495f4638; no voters:
I20260810 06:44:23.078061 23472 raft_consensus.cc:2781] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:44:23.078228 23472 raft_consensus.cc:482] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:23.078331 23472 raft_consensus.cc:3037] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:23.080452 23472 raft_consensus.cc:504] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } }
I20260810 06:44:23.080989 23472 leader_election.cc:288] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [CANDIDATE]: Term 1 election: Requested vote from peers c8fb35a3e0384cdbad640fe4524933a8 (127.10.242.2:44043), a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667)
I20260810 06:44:23.081389 23105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b501738f1d4e4d7fb3df9e7b901bc5dc" candidate_uuid: "4f53f4935ebb4d38a826be0c54c257df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8fb35a3e0384cdbad640fe4524933a8"
I20260810 06:44:23.081436 23324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b501738f1d4e4d7fb3df9e7b901bc5dc" candidate_uuid: "4f53f4935ebb4d38a826be0c54c257df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ab00aad90243d29326e71b495f4638"
I20260810 06:44:23.081557 23105 raft_consensus.cc:3037] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:23.081614 23324 raft_consensus.cc:3037] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:23.083839 23324 raft_consensus.cc:2454] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f53f4935ebb4d38a826be0c54c257df in term 1.
I20260810 06:44:23.083905 23105 raft_consensus.cc:2454] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f53f4935ebb4d38a826be0c54c257df in term 1.
I20260810 06:44:23.084205 22753 leader_election.cc:302] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f53f4935ebb4d38a826be0c54c257df, a8ab00aad90243d29326e71b495f4638; no voters:
I20260810 06:44:23.084456 23472 raft_consensus.cc:2781] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:23.084885 23472 raft_consensus.cc:686] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 1 LEADER]: Becoming Leader. State: Replica: 4f53f4935ebb4d38a826be0c54c257df, State: Running, Role: LEADER
I20260810 06:44:23.085376 23472 consensus_queue.cc:237] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } }
I20260810 06:44:23.090764 21335 catalog_manager.cc:5495] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f53f4935ebb4d38a826be0c54c257df (127.10.242.1). New cstate: current_term: 1 leader_uuid: "4f53f4935ebb4d38a826be0c54c257df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } health_report { overall_health: UNKNOWN } } }
I20260810 06:44:23.159457 23456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:23.154643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.10.242.1:45429)
0810 06:44:23.154819 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:23.154824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:23.154841 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:44:23.155025 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:23.155029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:23.155109 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:23.155234 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:23.155555 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.156326 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.157615 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.158024 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.158115 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:23.158573 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:23.159053 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:23.159168 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:23.159204 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 06:44:23.159265 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:44:23.167234 23324 raft_consensus.cc:1261] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [term 1 FOLLOWER]: Refusing update from remote peer 4f53f4935ebb4d38a826be0c54c257df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:23.167258 23105 raft_consensus.cc:1261] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [term 1 FOLLOWER]: Refusing update from remote peer 4f53f4935ebb4d38a826be0c54c257df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:23.168179 23477 consensus_queue.cc:1035] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:23.168499 23472 consensus_queue.cc:1035] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:23.180918 23481 mvcc.cc:205] Tried to move back new op lower bound from 7316866101923676160 to 7316866101601931264. Current Snapshot: MvccSnapshot[applied={T|T < 7316866101923676160}]
I20260810 06:44:23.200839 21335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:23.201109 21335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:23.207669 21335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6445ff13d22498e9dc52884364e67c0: Sending DeleteTablet for 3 replicas of tablet b501738f1d4e4d7fb3df9e7b901bc5dc
I20260810 06:44:23.208901 23292 tablet_service.cc:1514] Processing DeleteTablet for tablet b501738f1d4e4d7fb3df9e7b901bc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:23 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:52012
I20260810 06:44:23.209266 22826 tablet_service.cc:1514] Processing DeleteTablet for tablet b501738f1d4e4d7fb3df9e7b901bc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:23 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:44620
I20260810 06:44:23.209467 23068 tablet_service.cc:1514] Processing DeleteTablet for tablet b501738f1d4e4d7fb3df9e7b901bc5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:23 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:50604
I20260810 06:44:23.222026 23493 tablet_replica.cc:330] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: stopping tablet replica
I20260810 06:44:23.222036 23492 tablet_replica.cc:330] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: stopping tablet replica
I20260810 06:44:23.222043 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 21291
I20260810 06:44:23.222522 23493 raft_consensus.cc:2229] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:44:23.223027 21382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:23.223017 23492 raft_consensus.cc:2229] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:23.223587 23492 raft_consensus.cc:2258] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:23.225256 23494 tablet_replica.cc:330] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: stopping tablet replica
I20260810 06:44:23.225658 23494 raft_consensus.cc:2229] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:23.226174 23494 raft_consensus.cc:2258] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:23.227738 23494 ts_tablet_manager.cc:1905] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:23.229172 23493 raft_consensus.cc:2258] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:23.229338 23492 ts_tablet_manager.cc:1905] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:23.231205 23493 ts_tablet_manager.cc:1905] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:23.233744 23494 ts_tablet_manager.cc:1918] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:23.233897 23494 log.cc:1196] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/b501738f1d4e4d7fb3df9e7b901bc5dc
I20260810 06:44:23.234478 23494 ts_tablet_manager.cc:1939] T b501738f1d4e4d7fb3df9e7b901bc5dc P c8fb35a3e0384cdbad640fe4524933a8: Deleting consensus metadata
I20260810 06:44:23.237527 23493 ts_tablet_manager.cc:1918] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:23.237694 23493 log.cc:1196] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/b501738f1d4e4d7fb3df9e7b901bc5dc
I20260810 06:44:23.238334 23493 ts_tablet_manager.cc:1939] T b501738f1d4e4d7fb3df9e7b901bc5dc P 4f53f4935ebb4d38a826be0c54c257df: Deleting consensus metadata
I20260810 06:44:23.242223 23492 ts_tablet_manager.cc:1918] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:23.242434 23492 log.cc:1196] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/b501738f1d4e4d7fb3df9e7b901bc5dc
I20260810 06:44:23.242973 23492 ts_tablet_manager.cc:1939] T b501738f1d4e4d7fb3df9e7b901bc5dc P a8ab00aad90243d29326e71b495f4638: Deleting consensus metadata
I20260810 06:44:23.244951 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:23.212150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34788 (local address 127.10.242.61:44177)
0810 06:44:23.212345 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:23.212353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:23.221285 (+ 8932us) server_negotiation.cc:407] Connection header received
0810 06:44:23.222754 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:23.222760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:23.222853 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:23.222921 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:23.224804 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.225911 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.231687 (+ 5776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.232395 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.232500 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:23.237363 (+ 4863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:23.237370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:23.237374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:23.237415 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:23.241501 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:23.242422 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:23.242444 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:23.242448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:23.242611 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:23.243113 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:23.243119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:23.243122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:23.243499 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:23.243724 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:23.244369 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 06:44:23.244614 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:44:23.261917 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:23.221368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.10.242.60:41639)
0810 06:44:23.221528 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:23.221534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:23.246005 (+ 24471us) server_negotiation.cc:407] Connection header received
0810 06:44:23.246328 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:23.246335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:23.246438 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:23.246510 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:23.248570 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.249656 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.251522 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.252156 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.252277 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:23.255508 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:23.255516 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:23.255520 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:23.255555 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:23.259031 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:23.259776 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:23.259780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:23.259783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:23.259931 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:23.260337 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:23.260342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:23.260345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:23.260657 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:23.260905 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:23.261232 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 06:44:23.261450 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:44:23.366720 21316 catalog_manager.cc:4841] TS a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667): tablet b501738f1d4e4d7fb3df9e7b901bc5dc (table test-table [id=23844f2d21e6440ba777d444b24d6e26]) successfully deleted
I20260810 06:44:23.366750 21318 catalog_manager.cc:4841] TS 4f53f4935ebb4d38a826be0c54c257df (127.10.242.1:45429): tablet b501738f1d4e4d7fb3df9e7b901bc5dc (table test-table [id=23844f2d21e6440ba777d444b24d6e26]) successfully deleted
I20260810 06:44:23.367434 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 21291
I20260810 06:44:23.416910 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:23.401253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.10.242.61:44177)
0810 06:44:23.401443 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:23.401450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:23.402392 (+ 942us) server_negotiation.cc:407] Connection header received
0810 06:44:23.402610 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:23.402617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:23.402704 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:23.402817 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:23.404714 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.405486 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.407401 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.407987 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.408108 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:23.410726 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:23.410730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:23.410734 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:23.410767 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:23.413981 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:23.414784 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:23.414790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:23.414793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:23.414943 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:23.415311 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:23.415316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:23.415320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:23.415644 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:23.415896 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:23.416218 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 06:44:23.416590 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:44:23.420202 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:23.402516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.10.242.60:41639)
0810 06:44:23.402732 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:23.402738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:23.403182 (+ 444us) server_negotiation.cc:407] Connection header received
0810 06:44:23.403366 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:23.403372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:23.403451 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:23.403562 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:23.405547 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.406412 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.407823 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:23.408438 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:23.408559 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:23.411631 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:23.411637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:23.411642 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:23.411680 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:23.414829 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:23.415560 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:23.415565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:23.415568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:23.415710 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:23.418475 (+ 2765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:23.418481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:23.418484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:23.418829 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:23.419071 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:23.419654 (+ 583us) server_negotiation.cc:299] Negotiation successful
0810 06:44:23.419861 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:44:24.191511 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.190708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.1:56347)
0810 06:44:24.191218 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.191348 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":276,"threads_started":1}
W20260810 06:44:24.192018 22945 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:34471 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:24.195111 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.194430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.3:34865)
0810 06:44:24.194838 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.194942 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":226,"threads_started":1}
W20260810 06:44:24.195456 23409 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:34471 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:24.198385 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.197614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:46415)
0810 06:44:24.198074 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.198222 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":237,"threads_started":1}
W20260810 06:44:24.198810 23167 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:34471 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:24.708369 23514 raft_consensus.cc:482] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f6445ff13d22498e9dc52884364e67c0)
I20260810 06:44:24.708555 23516 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f6445ff13d22498e9dc52884364e67c0)
I20260810 06:44:24.708678 23514 raft_consensus.cc:504] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } }
I20260810 06:44:24.708792 23516 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } }
I20260810 06:44:24.710040 23514 leader_election.cc:288] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471), ab293efd178e4e7698d549af49df8d67 (127.10.242.61:44177)
I20260810 06:44:24.711206 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.710120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46614)
0810 06:44:24.710762 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.710951 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":317,"threads_started":1}
W20260810 06:44:24.711470 22641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:24.712479 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.710206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46624)
0810 06:44:24.711910 (+ 1704us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.712179 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1522,"thread_start_us":1299,"threads_started":1}
I20260810 06:44:24.713133 23516 leader_election.cc:288] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471), f77c30f7f2554020b85fa930af9287c5 (127.10.242.60:41639)
W20260810 06:44:24.713701 22293 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:24.720782 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.710482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:44177 (local address 127.0.0.1:34808)
0810 06:44:24.711007 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.711025 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:24.711154 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:24.713649 (+ 2495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:24.713655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:24.713690 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:24.714105 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:24.714118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:24.716072 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:24.716079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:24.718360 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:24.718377 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:24.720033 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:24.720040 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:24.720227 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:24.720265 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 06:44:24.720331 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:44:24.720458 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":808,"threads_started":1}
I20260810 06:44:24.721065 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.720509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46636)
0810 06:44:24.720681 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.720858 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 06:44:24.721254 22293 leader_election.cc:334] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471): Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:24.721374 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.720961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46650)
0810 06:44:24.721160 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.721247 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260810 06:44:24.721772 22641 leader_election.cc:334] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471): Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:24.721948 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.712528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41639 (local address 127.0.0.1:48378)
0810 06:44:24.712679 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.712697 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:24.712822 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:24.714634 (+ 1812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:24.714640 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:24.714669 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:24.714977 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:24.714988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:24.716738 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:24.716744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:24.718903 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:24.718920 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:24.721208 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:24.721215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:24.721424 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:24.721458 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:44:24.721547 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:44:24.721659 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"thread_start_us":886,"threads_started":1}
I20260810 06:44:24.721948 23521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.710865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34808 (local address 127.10.242.61:44177)
0810 06:44:24.713299 (+ 2434us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:24.713305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:24.713327 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:44:24.713391 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:24.713397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:24.713469 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:24.714293 (+ 824us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:24.714565 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:24.715882 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:24.718570 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:24.719817 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:24.720695 (+ 878us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:24.720796 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:24.721072 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 06:44:24.721554 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2305,"thread_start_us":2121,"threads_started":1}
I20260810 06:44:24.722551 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.712245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.10.242.60:41639)
0810 06:44:24.714030 (+ 1785us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:24.714036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:24.714064 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:44:24.714137 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:24.714141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:24.714241 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:24.714344 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:24.715177 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:24.716537 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:24.719419 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:24.720681 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:24.722010 (+ 1329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:24.722106 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:24.722187 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 06:44:24.722277 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1647,"thread_start_us":1822,"threads_started":1}
I20260810 06:44:24.722590 22335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f77c30f7f2554020b85fa930af9287c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab293efd178e4e7698d549af49df8d67" is_pre_election: true
I20260810 06:44:24.722921 22335 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f77c30f7f2554020b85fa930af9287c5 in term 1.
I20260810 06:44:24.723132 22671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab293efd178e4e7698d549af49df8d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f77c30f7f2554020b85fa930af9287c5" is_pre_election: true
I20260810 06:44:24.723459 22671 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab293efd178e4e7698d549af49df8d67 in term 1.
I20260810 06:44:24.723434 22635 leader_election.cc:302] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ab293efd178e4e7698d549af49df8d67, f77c30f7f2554020b85fa930af9287c5; no voters: f6445ff13d22498e9dc52884364e67c0
I20260810 06:44:24.723821 23514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 06:44:24.723973 23514 raft_consensus.cc:482] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f6445ff13d22498e9dc52884364e67c0)
I20260810 06:44:24.724103 23514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:44:24.724159 22293 leader_election.cc:302] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ab293efd178e4e7698d549af49df8d67, f77c30f7f2554020b85fa930af9287c5; no voters: f6445ff13d22498e9dc52884364e67c0
I20260810 06:44:24.724460 23516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 06:44:24.724586 23516 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f6445ff13d22498e9dc52884364e67c0)
I20260810 06:44:24.724677 23516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:44:24.726329 23514 raft_consensus.cc:504] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } }
I20260810 06:44:24.726850 23514 leader_election.cc:288] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [CANDIDATE]: Term 2 election: Requested vote from peers f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471), ab293efd178e4e7698d549af49df8d67 (127.10.242.61:44177)
I20260810 06:44:24.726786 23516 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } }
I20260810 06:44:24.727308 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.726972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46666)
0810 06:44:24.727102 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.727178 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 06:44:24.727329 22335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f77c30f7f2554020b85fa930af9287c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab293efd178e4e7698d549af49df8d67"
I20260810 06:44:24.727442 23516 leader_election.cc:288] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 2 election: Requested vote from peers f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471), f77c30f7f2554020b85fa930af9287c5 (127.10.242.60:41639)
I20260810 06:44:24.727613 22335 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f77c30f7f2554020b85fa930af9287c5 in current term 2: Already voted for candidate ab293efd178e4e7698d549af49df8d67 in this term.
I20260810 06:44:24.727954 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.727607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46674)
0810 06:44:24.727739 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.727816 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:44:24.728273 22671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab293efd178e4e7698d549af49df8d67" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f77c30f7f2554020b85fa930af9287c5"
I20260810 06:44:24.728554 22671 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ab293efd178e4e7698d549af49df8d67 in current term 2: Already voted for candidate f77c30f7f2554020b85fa930af9287c5 in this term.
I20260810 06:44:24.728631 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.728234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46684)
0810 06:44:24.728378 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.728456 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:44:24.728785 22641 leader_election.cc:334] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471): Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:24.728914 22641 leader_election.cc:302] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f77c30f7f2554020b85fa930af9287c5; no voters: ab293efd178e4e7698d549af49df8d67, f6445ff13d22498e9dc52884364e67c0
I20260810 06:44:24.729179 23514 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 06:44:24.729717 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:24.729374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46688)
0810 06:44:24.729525 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:24.729607 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 06:44:24.729858 22293 leader_election.cc:334] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471): Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:24.729992 22293 leader_election.cc:302] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab293efd178e4e7698d549af49df8d67; no voters: f6445ff13d22498e9dc52884364e67c0, f77c30f7f2554020b85fa930af9287c5
I20260810 06:44:24.730247 23516 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 06:44:25.193274 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:25.192584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.1:37101)
0810 06:44:25.192998 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:25.193090 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":220,"threads_started":1}
I20260810 06:44:25.196597 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:25.195972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.3:54413)
0810 06:44:25.196359 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:25.196435 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":209,"threads_started":1}
I20260810 06:44:25.200207 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:25.199480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:47843)
0810 06:44:25.199942 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:25.200029 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":241,"threads_started":1}
I20260810 06:44:26.195627 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.194552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.1:53569)
0810 06:44:26.195182 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.195321 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":283,"threads_started":1}
I20260810 06:44:26.198868 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.197805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.3:39349)
0810 06:44:26.198438 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.198586 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":380,"threads_started":1}
I20260810 06:44:26.202469 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.201356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:52913)
0810 06:44:26.202055 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.202242 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":416,"threads_started":1}
I20260810 06:44:26.279573 23540 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:26.279963 23540 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } }
I20260810 06:44:26.281528 23540 leader_election.cc:288] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471), f77c30f7f2554020b85fa930af9287c5 (127.10.242.60:41639)
I20260810 06:44:26.283774 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.281847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46906)
0810 06:44:26.283074 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.283308 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":958,"thread_start_us":654,"threads_started":1}
I20260810 06:44:26.284219 22671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab293efd178e4e7698d549af49df8d67" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f77c30f7f2554020b85fa930af9287c5" is_pre_election: true
I20260810 06:44:26.284694 22671 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab293efd178e4e7698d549af49df8d67 in term 2.
I20260810 06:44:26.285468 22293 leader_election.cc:302] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab293efd178e4e7698d549af49df8d67, f77c30f7f2554020b85fa930af9287c5; no voters:
I20260810 06:44:26.285899 23540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 06:44:26.286132 23540 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:26.286362 23540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:44:26.286803 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.286191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46908)
0810 06:44:26.286439 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.286604 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 06:44:26.287170 22293 leader_election.cc:334] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471): Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:26.290411 23540 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } }
I20260810 06:44:26.291393 23540 leader_election.cc:288] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 3 election: Requested vote from peers f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471), f77c30f7f2554020b85fa930af9287c5 (127.10.242.60:41639)
I20260810 06:44:26.292261 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.291528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46914)
0810 06:44:26.291808 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.291975 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 06:44:26.293412 22671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab293efd178e4e7698d549af49df8d67" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f77c30f7f2554020b85fa930af9287c5"
I20260810 06:44:26.293905 22671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:44:26.297683 22671 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab293efd178e4e7698d549af49df8d67 in term 3.
I20260810 06:44:26.298434 22293 leader_election.cc:302] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab293efd178e4e7698d549af49df8d67, f77c30f7f2554020b85fa930af9287c5; no voters:
I20260810 06:44:26.298720 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.298088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46928)
0810 06:44:26.298292 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.298455 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:44:26.298939 22293 leader_election.cc:334] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471): Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
I20260810 06:44:26.299075 23540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:44:26.299803 23540 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [term 3 LEADER]: Becoming Leader. State: Replica: ab293efd178e4e7698d549af49df8d67, State: Running, Role: LEADER
I20260810 06:44:26.300462 23540 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } }
I20260810 06:44:26.303146 23545 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab293efd178e4e7698d549af49df8d67. Latest consensus state: current_term: 3 leader_uuid: "ab293efd178e4e7698d549af49df8d67" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:26.303741 23545 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:26.304697 23548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:26.310024 23548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:26.311422 23548 catalog_manager.cc:1259] Loaded cluster ID: 1e4c471737e54d9c8f57671acd4cdbcc
I20260810 06:44:26.311528 23548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:26.313501 23548 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:26.314914 23548 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Loaded TSK: 0
I20260810 06:44:26.316468 23548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:44:26.389225 22315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:44:26.402108 22315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:44:26.409111 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.408506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46944)
0810 06:44:26.408794 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.408963 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 06:44:26.409518 22671 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [term 3 FOLLOWER]: Refusing update from remote peer ab293efd178e4e7698d549af49df8d67: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 06:44:26.410073 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.409728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46950)
0810 06:44:26.409880 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.409963 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:44:26.410188 23545 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 06:44:26.410482 22293 consensus_peers.cc:487] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 -> Peer f6445ff13d22498e9dc52884364e67c0 (127.10.242.62:34471): Couldn't send request to peer f6445ff13d22498e9dc52884364e67c0. Status: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:44:26.412508 23552 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab293efd178e4e7698d549af49df8d67. Latest consensus state: current_term: 3 leader_uuid: "ab293efd178e4e7698d549af49df8d67" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:26.412755 23552 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:44:26.414214 23540 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ab293efd178e4e7698d549af49df8d67" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:26.414487 23540 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:26.414878 23552 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ab293efd178e4e7698d549af49df8d67" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f6445ff13d22498e9dc52884364e67c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34471 } } peers { permanent_uuid: "ab293efd178e4e7698d549af49df8d67" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 44177 } attrs { promote: false } } peers { permanent_uuid: "f77c30f7f2554020b85fa930af9287c5" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41639 } attrs { promote: false } } }
I20260810 06:44:26.415135 23552 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77c30f7f2554020b85fa930af9287c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:44:26.430701 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.422377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:44043 (local address 127.0.0.1:50614)
0810 06:44:26.422511 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.422529 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:26.422640 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:26.423637 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:26.423642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:26.423670 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:26.424043 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.424056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.425752 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.425758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.428018 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.428035 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.429795 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.429803 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.430019 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.430061 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 06:44:26.430140 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:44:26.430311 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:44:26.431877 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.422647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.10.242.2:44043)
0810 06:44:26.423277 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:26.423283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:26.423303 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:44:26.423375 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:26.423380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:26.423469 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:26.424210 (+ 741us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:26.424411 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.425583 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.428253 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.429580 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.431192 (+ 1612us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.431314 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:26.431369 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 06:44:26.431519 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":256,"threads_started":1}
I20260810 06:44:26.432690 23068 tablet_service.cc:1467] Processing CreateTablet for tablet 659200f250194b86a8a15e50c6cab4ab (DEFAULT_TABLE table=test-table [id=7bab41f5168a40d8bc454d3fd937b708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:26.433212 23068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 659200f250194b86a8a15e50c6cab4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:26.440253 23573 tablet_bootstrap.cc:492] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: Bootstrap starting.
I20260810 06:44:26.442615 23573 tablet_bootstrap.cc:654] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:26.445367 23573 tablet_bootstrap.cc:492] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: No bootstrap required, opened a new log
I20260810 06:44:26.445538 23573 ts_tablet_manager.cc:1397] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 06:44:26.446365 23573 raft_consensus.cc:348] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } }
I20260810 06:44:26.446628 23573 raft_consensus.cc:374] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:26.446718 23573 raft_consensus.cc:729] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8fb35a3e0384cdbad640fe4524933a8, State: Initialized, Role: FOLLOWER
I20260810 06:44:26.447007 23573 consensus_queue.cc:260] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } }
I20260810 06:44:26.448490 23573 ts_tablet_manager.cc:1428] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:26.449229 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.448680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:48981)
0810 06:44:26.448999 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.449087 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214}
I20260810 06:44:26.451615 23170 heartbeater.cc:503] Master 127.10.242.61:44177 was elected leader, sending a full tablet report...
I20260810 06:44:26.452087 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.451381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:50607)
0810 06:44:26.451843 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.451925 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361}
I20260810 06:44:26.453363 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.423457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52026 (local address 127.10.242.3:38667)
0810 06:44:26.444126 (+ 20669us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:26.444133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:26.444156 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:44:26.444252 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:26.444258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:26.444338 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:26.444446 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:26.446310 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.448600 (+ 2290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.451148 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.452279 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.452406 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.452513 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:26.452923 (+ 410us) server_negotiation.cc:299] Negotiation successful
0810 06:44:26.453064 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20540,"thread_start_us":20457,"threads_started":1}
I20260810 06:44:26.453361 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.422772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38667 (local address 127.0.0.1:52026)
0810 06:44:26.430552 (+ 7780us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.430573 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:26.430700 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:26.444506 (+ 13806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:26.444512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:26.444542 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:26.444959 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.444969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.448765 (+ 3796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.448771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.450925 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.450940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.452475 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.452482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.452731 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.452766 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:44:26.452858 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 06:44:26.452983 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7662,"mutex_wait_us":20,"thread_start_us":7535,"threads_started":1}
I20260810 06:44:26.454341 23292 tablet_service.cc:1467] Processing CreateTablet for tablet 659200f250194b86a8a15e50c6cab4ab (DEFAULT_TABLE table=test-table [id=7bab41f5168a40d8bc454d3fd937b708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:26.454818 23292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 659200f250194b86a8a15e50c6cab4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:26.456416 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.423094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45429 (local address 127.0.0.1:44640)
0810 06:44:26.430899 (+ 7805us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.430914 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:26.431001 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:26.449930 (+ 18929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:26.449935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:26.449959 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:26.450136 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.450148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.451779 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.451784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.453853 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.453889 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.455731 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.455737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.455898 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.455921 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:44:26.455987 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:44:26.456102 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7729,"mutex_wait_us":207,"thread_start_us":24359,"threads_started":1}
I20260810 06:44:26.456475 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.423253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44640 (local address 127.10.242.1:45429)
0810 06:44:26.449615 (+ 26362us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:26.449621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:26.449642 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:44:26.449712 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:26.449716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:26.449787 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:26.450316 (+ 529us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:26.450521 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.451623 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.454311 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.455586 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.455707 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.455805 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:26.456047 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 06:44:26.456176 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26226,"thread_start_us":29890,"threads_started":1}
I20260810 06:44:26.457262 22826 tablet_service.cc:1467] Processing CreateTablet for tablet 659200f250194b86a8a15e50c6cab4ab (DEFAULT_TABLE table=test-table [id=7bab41f5168a40d8bc454d3fd937b708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:26.457697 22826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 659200f250194b86a8a15e50c6cab4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:26.460490 23577 tablet_bootstrap.cc:492] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: Bootstrap starting.
I20260810 06:44:26.461866 23577 tablet_bootstrap.cc:654] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:26.463605 23577 tablet_bootstrap.cc:492] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: No bootstrap required, opened a new log
I20260810 06:44:26.463757 23577 ts_tablet_manager.cc:1397] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 06:44:26.463771 23582 tablet_bootstrap.cc:492] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: Bootstrap starting.
I20260810 06:44:26.464491 23577 raft_consensus.cc:348] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } }
I20260810 06:44:26.465000 23577 raft_consensus.cc:374] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:26.465106 23577 raft_consensus.cc:729] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ab00aad90243d29326e71b495f4638, State: Initialized, Role: FOLLOWER
I20260810 06:44:26.465260 23582 tablet_bootstrap.cc:654] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:26.465334 23577 consensus_queue.cc:260] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } }
I20260810 06:44:26.466079 23577 ts_tablet_manager.cc:1428] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:44:26.467054 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.466454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.3:45155)
0810 06:44:26.466804 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.466907 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237}
I20260810 06:44:26.469107 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.468699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.3:45097)
0810 06:44:26.468852 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.468935 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:44:26.469164 23414 heartbeater.cc:503] Master 127.10.242.61:44177 was elected leader, sending a full tablet report...
I20260810 06:44:26.470378 23582 tablet_bootstrap.cc:492] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: No bootstrap required, opened a new log
I20260810 06:44:26.470525 23582 ts_tablet_manager.cc:1397] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260810 06:44:26.471182 23582 raft_consensus.cc:348] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } }
I20260810 06:44:26.471362 23582 raft_consensus.cc:374] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:26.471449 23582 raft_consensus.cc:729] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f53f4935ebb4d38a826be0c54c257df, State: Initialized, Role: FOLLOWER
I20260810 06:44:26.471662 23582 consensus_queue.cc:260] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } }
I20260810 06:44:26.472380 23582 ts_tablet_manager.cc:1428] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:44:26.472978 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.472603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.1:59855)
0810 06:44:26.472764 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.472849 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:44:26.473906 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.473539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.1:57887)
0810 06:44:26.473681 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.473762 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:44:26.474323 22947 heartbeater.cc:503] Master 127.10.242.61:44177 was elected leader, sending a full tablet report...
I20260810 06:44:26.522468 23575 raft_consensus.cc:482] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:26.522670 23575 raft_consensus.cc:504] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } }
I20260810 06:44:26.524116 23575 leader_election.cc:288] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667), 4f53f4935ebb4d38a826be0c54c257df (127.10.242.1:45429)
I20260810 06:44:26.530793 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.524496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:59065 (local address 127.10.242.3:38667)
0810 06:44:26.524720 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:26.524726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:26.524744 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:44:26.524813 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:26.524818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:26.524889 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:26.524981 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:26.525576 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.526587 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.528926 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.529993 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.530172 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.530258 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:26.530464 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 06:44:26.530588 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:44:26.530850 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.524332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38667 (local address 127.10.242.2:59065)
0810 06:44:26.524472 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.524491 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:26.524610 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:26.525036 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:26.525041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:26.525073 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:26.525426 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.525438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.526722 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.526726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.528758 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.528777 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.530189 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.530195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.530318 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.530343 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:44:26.530416 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:44:26.530534 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":4}
I20260810 06:44:26.531344 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.524508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:46573 (local address 127.10.242.1:45429)
0810 06:44:26.524663 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:26.524667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:26.525485 (+ 818us) server_negotiation.cc:407] Connection header received
0810 06:44:26.525661 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:26.525666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:26.525737 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:26.525826 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:26.526223 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.527030 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.529228 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.530407 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.530554 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.530616 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:26.531005 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 06:44:26.531132 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:44:26.531400 23324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "659200f250194b86a8a15e50c6cab4ab" candidate_uuid: "c8fb35a3e0384cdbad640fe4524933a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ab00aad90243d29326e71b495f4638" is_pre_election: true
I20260810 06:44:26.531651 23324 raft_consensus.cc:2454] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8fb35a3e0384cdbad640fe4524933a8 in term 0.
I20260810 06:44:26.531999 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.524337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45429 (local address 127.10.242.2:46573)
0810 06:44:26.525366 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.525387 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:26.525536 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:26.525831 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:26.525834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:26.525852 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:26.526086 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.526093 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.527144 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.527149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.529063 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:26.529073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.530612 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.530619 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:26.530808 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.530844 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:44:26.531513 (+ 669us) client_negotiation.cc:240] Negotiation successful
0810 06:44:26.531630 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":431,"thread_start_us":293,"threads_started":1}
I20260810 06:44:26.532212 22994 leader_election.cc:302] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8ab00aad90243d29326e71b495f4638, c8fb35a3e0384cdbad640fe4524933a8; no voters:
I20260810 06:44:26.532596 23575 raft_consensus.cc:2781] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:44:26.532734 23575 raft_consensus.cc:482] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:26.532838 23575 raft_consensus.cc:3037] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:26.532737 22854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "659200f250194b86a8a15e50c6cab4ab" candidate_uuid: "c8fb35a3e0384cdbad640fe4524933a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f53f4935ebb4d38a826be0c54c257df" is_pre_election: true
I20260810 06:44:26.533138 22854 raft_consensus.cc:2454] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8fb35a3e0384cdbad640fe4524933a8 in term 0.
I20260810 06:44:26.535126 23575 raft_consensus.cc:504] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } }
I20260810 06:44:26.535645 23575 leader_election.cc:288] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [CANDIDATE]: Term 1 election: Requested vote from peers a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667), 4f53f4935ebb4d38a826be0c54c257df (127.10.242.1:45429)
I20260810 06:44:26.536016 22854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "659200f250194b86a8a15e50c6cab4ab" candidate_uuid: "c8fb35a3e0384cdbad640fe4524933a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f53f4935ebb4d38a826be0c54c257df"
I20260810 06:44:26.536026 23324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "659200f250194b86a8a15e50c6cab4ab" candidate_uuid: "c8fb35a3e0384cdbad640fe4524933a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8ab00aad90243d29326e71b495f4638"
I20260810 06:44:26.536172 22854 raft_consensus.cc:3037] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:26.536197 23324 raft_consensus.cc:3037] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:26.538473 22854 raft_consensus.cc:2454] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8fb35a3e0384cdbad640fe4524933a8 in term 1.
I20260810 06:44:26.538846 22996 leader_election.cc:302] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f53f4935ebb4d38a826be0c54c257df, c8fb35a3e0384cdbad640fe4524933a8; no voters:
I20260810 06:44:26.539095 23575 raft_consensus.cc:2781] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:26.539088 23324 raft_consensus.cc:2454] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8fb35a3e0384cdbad640fe4524933a8 in term 1.
I20260810 06:44:26.539733 23575 raft_consensus.cc:686] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 1 LEADER]: Becoming Leader. State: Replica: c8fb35a3e0384cdbad640fe4524933a8, State: Running, Role: LEADER
I20260810 06:44:26.540104 23575 consensus_queue.cc:237] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } }
I20260810 06:44:26.540714 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.540246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:54203)
0810 06:44:26.540394 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.540488 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:44:26.543787 22315 catalog_manager.cc:5495] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8fb35a3e0384cdbad640fe4524933a8 (127.10.242.2). New cstate: current_term: 1 leader_uuid: "c8fb35a3e0384cdbad640fe4524933a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8fb35a3e0384cdbad640fe4524933a8" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 44043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 } health_report { overall_health: UNKNOWN } } }
I20260810 06:44:26.561892 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.557068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.10.242.2:44043)
0810 06:44:26.557205 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:26.557209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:26.557225 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:44:26.557413 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:26.557417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:26.557484 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:26.557634 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:26.558045 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.558949 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.560201 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.560587 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.560675 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.560971 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:26.561502 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:26.561612 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:26.561650 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 06:44:26.561710 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:44:26.565665 23324 raft_consensus.cc:1261] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [term 1 FOLLOWER]: Refusing update from remote peer c8fb35a3e0384cdbad640fe4524933a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:26.565917 22854 raft_consensus.cc:1261] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [term 1 FOLLOWER]: Refusing update from remote peer c8fb35a3e0384cdbad640fe4524933a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:26.566375 23590 consensus_queue.cc:1035] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8ab00aad90243d29326e71b495f4638" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:26.566803 23575 consensus_queue.cc:1035] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f53f4935ebb4d38a826be0c54c257df" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:26.568831 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.568386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.3:52097)
0810 06:44:26.568565 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.568667 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:44:26.572170 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.571727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.1:47513)
0810 06:44:26.571901 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.572001 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 06:44:26.574546 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.574113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:53119)
0810 06:44:26.574284 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.574379 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:44:26.576041 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.575588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.1:44227)
0810 06:44:26.575767 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.575866 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 06:44:26.577571 23597 mvcc.cc:205] Tried to move back new op lower bound from 7316866115846283264 to 7316866115752288256. Current Snapshot: MvccSnapshot[applied={T|T < 7316866115846283264 or (T in {7316866115846283264})}]
I20260810 06:44:26.589975 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.589334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.2:47307)
0810 06:44:26.589702 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.589813 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237}
I20260810 06:44:26.591915 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.591318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.10.242.3:60187)
0810 06:44:26.591635 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.591724 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260810 06:44:26.595052 22316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:26.595377 22316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:26.600927 22316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab293efd178e4e7698d549af49df8d67: Sending DeleteTablet for 3 replicas of tablet 659200f250194b86a8a15e50c6cab4ab
I20260810 06:44:26.601830 23292 tablet_service.cc:1514] Processing DeleteTablet for tablet 659200f250194b86a8a15e50c6cab4ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:26 UTC) from {username='oryx', principal='oryx/127.10.242.61@KRBTEST.COM'} at 127.0.0.1:52026
I20260810 06:44:26.602073 22826 tablet_service.cc:1514] Processing DeleteTablet for tablet 659200f250194b86a8a15e50c6cab4ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:26 UTC) from {username='oryx', principal='oryx/127.10.242.61@KRBTEST.COM'} at 127.0.0.1:44640
I20260810 06:44:26.602538 23610 tablet_replica.cc:330] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: stopping tablet replica
I20260810 06:44:26.602741 23610 raft_consensus.cc:2229] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:26.602820 23612 tablet_replica.cc:330] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: stopping tablet replica
I20260810 06:44:26.602954 23068 tablet_service.cc:1514] Processing DeleteTablet for tablet 659200f250194b86a8a15e50c6cab4ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:26 UTC) from {username='oryx', principal='oryx/127.10.242.61@KRBTEST.COM'} at 127.0.0.1:50614
I20260810 06:44:26.603036 23610 raft_consensus.cc:2258] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:26.603209 23612 raft_consensus.cc:2229] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:26.603543 23612 raft_consensus.cc:2258] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:26.603677 23617 tablet_replica.cc:330] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: stopping tablet replica
I20260810 06:44:26.604525 23617 raft_consensus.cc:2229] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:44:26.605139 23612 ts_tablet_manager.cc:1905] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:26.605352 23617 raft_consensus.cc:2258] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:26.605501 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 22619
I20260810 06:44:26.606436 23617 ts_tablet_manager.cc:1905] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:26.608294 23610 ts_tablet_manager.cc:1905] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:26.612412 23610 ts_tablet_manager.cc:1918] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:26.612512 23610 log.cc:1196] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/659200f250194b86a8a15e50c6cab4ab
I20260810 06:44:26.612831 23610 ts_tablet_manager.cc:1939] T 659200f250194b86a8a15e50c6cab4ab P a8ab00aad90243d29326e71b495f4638: Deleting consensus metadata
I20260810 06:44:26.612986 23612 ts_tablet_manager.cc:1918] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:26.613163 23612 log.cc:1196] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/659200f250194b86a8a15e50c6cab4ab
I20260810 06:44:26.613672 23612 ts_tablet_manager.cc:1939] T 659200f250194b86a8a15e50c6cab4ab P 4f53f4935ebb4d38a826be0c54c257df: Deleting consensus metadata
I20260810 06:44:26.613684 22288 catalog_manager.cc:4841] TS a8ab00aad90243d29326e71b495f4638 (127.10.242.3:38667): tablet 659200f250194b86a8a15e50c6cab4ab (table test-table [id=7bab41f5168a40d8bc454d3fd937b708]) successfully deleted
I20260810 06:44:26.614830 22290 catalog_manager.cc:4841] TS 4f53f4935ebb4d38a826be0c54c257df (127.10.242.1:45429): tablet 659200f250194b86a8a15e50c6cab4ab (table test-table [id=7bab41f5168a40d8bc454d3fd937b708]) successfully deleted
I20260810 06:44:26.619029 23617 ts_tablet_manager.cc:1918] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:26.619174 23617 log.cc:1196] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/659200f250194b86a8a15e50c6cab4ab
I20260810 06:44:26.619581 23617 ts_tablet_manager.cc:1939] T 659200f250194b86a8a15e50c6cab4ab P c8fb35a3e0384cdbad640fe4524933a8: Deleting consensus metadata
I20260810 06:44:26.620446 22293 catalog_manager.cc:4841] TS c8fb35a3e0384cdbad640fe4524933a8 (127.10.242.2:44043): tablet 659200f250194b86a8a15e50c6cab4ab (table test-table [id=7bab41f5168a40d8bc454d3fd937b708]) successfully deleted
I20260810 06:44:26.632442 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.606259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.10.242.1:45429)
0810 06:44:26.606478 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:26.606485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:26.618646 (+ 12161us) server_negotiation.cc:407] Connection header received
0810 06:44:26.618838 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:26.618843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:26.618924 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:26.619040 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:26.620931 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.621675 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.623436 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.623808 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.623898 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.627120 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:26.627138 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:26.627154 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:26.627183 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:26.629803 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:26.630638 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:26.630646 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:26.630648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:26.630770 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:26.631146 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:26.631149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:26.631151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:26.631466 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:26.631689 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:26.632001 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 06:44:26.632179 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260810 06:44:26.633071 22935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:26.751398 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 22619
I20260810 06:44:26.781649 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 22962
I20260810 06:44:26.795722 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.782669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.10.242.2:44043)
0810 06:44:26.782847 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:26.782852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:26.783165 (+ 313us) server_negotiation.cc:407] Connection header received
0810 06:44:26.783336 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:26.783342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:26.783450 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:26.783544 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:26.784851 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.785669 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.786985 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.787422 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.787534 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.790056 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:26.790069 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:26.790085 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:26.790147 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:26.792954 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:26.793601 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:26.793612 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:26.793615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:26.793773 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:26.794117 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:26.794130 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:26.794133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:26.794575 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:26.794887 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:26.795181 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 06:44:26.795395 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:44:26.796371 23160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:26.910519 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 22962
I20260810 06:44:26.926898 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.926443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46966)
0810 06:44:26.926605 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.926719 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 06:44:26.928139 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.927814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34471 (local address 127.0.0.1:46974)
0810 06:44:26.927949 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:26.928019 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:44:26.939052 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 23187
I20260810 06:44:26.951601 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:26.939758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52042 (local address 127.10.242.3:38667)
0810 06:44:26.939936 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:26.939942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:26.939962 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:44:26.940119 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:26.940134 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:26.940198 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:26.940280 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:26.941646 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.942493 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.943684 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:26.944105 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:26.944202 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:26.946941 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:26.946946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:26.946949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:26.946980 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:26.949485 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:26.950055 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:26.950058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:26.950060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:26.950235 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:26.950534 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:26.950537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:26.950539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:26.950807 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:26.950991 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:26.951188 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 06:44:26.951346 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:44:26.952215 23402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:27.043088 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 23187
I20260810 06:44:27.069826 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 22089
I20260810 06:44:27.070904 22364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:27.194929 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 22089
I20260810 06:44:27.224934 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 22259
I20260810 06:44:27.226045 22703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:27.342206 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 22259
2026-08-10T06:44:27Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11245 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:44:27 dist-test-slave-zj8q krb5kdc[23632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:44:27 dist-test-slave-zj8q krb5kdc[23632](info): set up 2 sockets
Aug 10 06:44:27 dist-test-slave-zj8q krb5kdc[23632](info): commencing operation
krb5kdc: starting...
W20260810 06:44:28.593575 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:44:28 dist-test-slave-zj8q krb5kdc[23632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:44:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:28Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.62.keytab.
Entry for principal oryx/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.62.keytab.
I20260810 06:44:28.813012 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40355
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.62.keytab
--principal=oryx/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:40355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:29.136756 23653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:29.137045 23653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:29.137176 23653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:29.144121 23653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:29.144217 23653 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:44:29.144279 23653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:29.144362 23653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:29.144423 23653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:29.144506 23653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:29.154505 23653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:40355
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.62.keytab
--principal=oryx/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:29.155751 23653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:29.157038 23653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:29.168565 23672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:29.168599 23667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:29.168504 23668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:29.169258 23653 server_base.cc:1034] running on GCE node
I20260810 06:44:29.170009 23653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:29.171351 23653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:29.172569 23653 hybrid_clock.cc:648] HybridClock initialized: now 1786344269172498 us; error 45 us; skew 500 ppm
I20260810 06:44:29.172885 23653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:29 dist-test-slave-zj8q krb5kdc[23632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344269, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:29.175927 23653 init.cc:377] Logged in from keytab as oryx/127.10.242.62@KRBTEST.COM (short username oryx)
I20260810 06:44:29.177311 23653 webserver.cc:460] Webserver started at http://127.10.242.62:40907/ using document root <none> and password file <none>
I20260810 06:44:29.177999 23653 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:29.178102 23653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:29.178434 23653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:29.181365 23653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "6e9087eee7a8481aa1aa73c445b96f0d"
format_stamp: "Formatted at 2026-08-10 06:44:29 on dist-test-slave-zj8q"
server_key: "262e095cf7136b1756efb6bb321ca8f2"
server_key_iv: "f775abf2646bf14edbe797660c66e99a"
server_key_version: "encryptionkey@0"
I20260810 06:44:29.182143 23653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "6e9087eee7a8481aa1aa73c445b96f0d"
format_stamp: "Formatted at 2026-08-10 06:44:29 on dist-test-slave-zj8q"
server_key: "262e095cf7136b1756efb6bb321ca8f2"
server_key_iv: "f775abf2646bf14edbe797660c66e99a"
server_key_version: "encryptionkey@0"
I20260810 06:44:29.187212 23653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:44:29.190608 23684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:29.191903 23653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 06:44:29.192062 23653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "6e9087eee7a8481aa1aa73c445b96f0d"
format_stamp: "Formatted at 2026-08-10 06:44:29 on dist-test-slave-zj8q"
server_key: "262e095cf7136b1756efb6bb321ca8f2"
server_key_iv: "f775abf2646bf14edbe797660c66e99a"
server_key_version: "encryptionkey@0"
I20260810 06:44:29.192262 23653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:29.205372 23653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:29.209054 23653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:29.209297 23653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:29.228593 23653 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:40355
I20260810 06:44:29.228617 23775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:40355 every 8 connection(s)
I20260810 06:44:29.229872 23653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:29.233413 23776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:29.237178 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 23653
I20260810 06:44:29.237407 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:29.238268 11208 external_mini_cluster.cc:1428] Setting key 0c042376dd39413d7cc59c91183682d8
I20260810 06:44:29.244890 23776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d: Bootstrap starting.
I20260810 06:44:29.248417 23776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:29.249578 23776 log.cc:824] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:29.252735 23776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d: No bootstrap required, opened a new log
Aug 10 06:44:29 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:29.261557 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:29.240373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.10.242.62:40355)
0810 06:44:29.241143 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:29.241155 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:29.241197 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:44:29.243376 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:29.243400 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:29.243758 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:29.244051 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:29.245899 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.247295 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.249041 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.249741 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.249873 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:29.254623 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:29.254634 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:29.254648 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:29.254693 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:29.257629 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:29.258467 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:29.258474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:29.258480 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:29.258662 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:29.259040 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:29.259043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:29.259044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:29.259574 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:29.259800 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:29.260246 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 06:44:29.260577 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":358,"threads_started":1}
I20260810 06:44:29.261439 23776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9087eee7a8481aa1aa73c445b96f0d" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40355 } }
I20260810 06:44:29.261793 23776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:29.261910 23776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9087eee7a8481aa1aa73c445b96f0d, State: Initialized, Role: FOLLOWER
I20260810 06:44:29.262501 23776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9087eee7a8481aa1aa73c445b96f0d" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40355 } }
I20260810 06:44:29.262749 23776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:29.262899 23776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:29.263092 23776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:29.265614 23776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9087eee7a8481aa1aa73c445b96f0d" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40355 } }
I20260810 06:44:29.266052 23776 leader_election.cc:302] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e9087eee7a8481aa1aa73c445b96f0d; no voters:
I20260810 06:44:29.266590 23776 leader_election.cc:288] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:29.266706 23781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:29.267354 23781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [term 1 LEADER]: Becoming Leader. State: Replica: 6e9087eee7a8481aa1aa73c445b96f0d, State: Running, Role: LEADER
I20260810 06:44:29.267879 23781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9087eee7a8481aa1aa73c445b96f0d" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40355 } }
I20260810 06:44:29.268467 23776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:29.270537 23785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e9087eee7a8481aa1aa73c445b96f0d. Latest consensus state: current_term: 1 leader_uuid: "6e9087eee7a8481aa1aa73c445b96f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9087eee7a8481aa1aa73c445b96f0d" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40355 } } }
I20260810 06:44:29.270562 23782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e9087eee7a8481aa1aa73c445b96f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9087eee7a8481aa1aa73c445b96f0d" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40355 } } }
I20260810 06:44:29.271056 23785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:29.271056 23782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:29.274921 23790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:29.279021 23790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:29.287043 23790 catalog_manager.cc:1347] Generated new cluster ID: 104cd31153b64a46b282395fb554aa03
I20260810 06:44:29.287153 23790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:29.322206 23790 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:29.323900 23790 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:29.329876 23790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d: Generated new TSK 0
I20260810 06:44:29.330717 23790 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab.
Entry for principal oryx/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab.
I20260810 06:44:29.391637 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40355
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab
--principal=oryx/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:29.717033 23824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:29.717310 23824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:29.717455 23824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:29.724454 23824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:29.724576 23824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:29.724704 23824 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:44:29.724813 23824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:44:29.736078 23824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab
--principal=oryx/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.62:40355
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:29.737476 23824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:29.739260 23824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:29.754014 23839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:29.754029 23840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:29.754426 23842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:29.755257 23824 server_base.cc:1034] running on GCE node
I20260810 06:44:29.755890 23824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:29.756857 23824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:29.758101 23824 hybrid_clock.cc:648] HybridClock initialized: now 1786344269758032 us; error 45 us; skew 500 ppm
I20260810 06:44:29.758419 23824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:29 dist-test-slave-zj8q krb5kdc[23632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344269, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:29.761816 23824 init.cc:377] Logged in from keytab as oryx/127.10.242.1@KRBTEST.COM (short username oryx)
I20260810 06:44:29.763890 23824 webserver.cc:460] Webserver started at http://127.10.242.1:34721/ using document root <none> and password file <none>
I20260810 06:44:29.764789 23824 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:29.764910 23824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:29.765287 23824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:29.769397 23824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "6a8adf2f1d144af59fa1868bd576e51d"
format_stamp: "Formatted at 2026-08-10 06:44:29 on dist-test-slave-zj8q"
server_key: "45632a9edef0b37c2daa2ddaf8ac5f7b"
server_key_iv: "000afe3dde4d8dfea313af97aade2102"
server_key_version: "encryptionkey@0"
I20260810 06:44:29.770471 23824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "6a8adf2f1d144af59fa1868bd576e51d"
format_stamp: "Formatted at 2026-08-10 06:44:29 on dist-test-slave-zj8q"
server_key: "45632a9edef0b37c2daa2ddaf8ac5f7b"
server_key_iv: "000afe3dde4d8dfea313af97aade2102"
server_key_version: "encryptionkey@0"
I20260810 06:44:29.777029 23824 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 06:44:29.781430 23853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:29.782840 23824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:29.783001 23824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "6a8adf2f1d144af59fa1868bd576e51d"
format_stamp: "Formatted at 2026-08-10 06:44:29 on dist-test-slave-zj8q"
server_key: "45632a9edef0b37c2daa2ddaf8ac5f7b"
server_key_iv: "000afe3dde4d8dfea313af97aade2102"
server_key_version: "encryptionkey@0"
I20260810 06:44:29.783205 23824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:29.798327 23824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:29.802330 23824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:29.802572 23824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:29.805186 23824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:29.805269 23824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:29.805401 23824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:29.805454 23824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:44:29 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344269, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:29.829102 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:29.805636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:39053 (local address 127.10.242.62:40355)
0810 06:44:29.806354 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:29.806361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:29.807487 (+ 1126us) server_negotiation.cc:407] Connection header received
0810 06:44:29.810333 (+ 2846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:29.810340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:29.810433 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:29.810557 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:29.812811 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.813828 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.816720 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.817278 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.817425 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:29.822061 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:29.822066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:29.822070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:29.822103 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:29.825132 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:29.826276 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:29.826282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:29.826286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:29.826418 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:29.827015 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:29.827020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:29.827023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:29.827340 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:29.827481 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:29.828451 (+ 970us) server_negotiation.cc:299] Negotiation successful
0810 06:44:29.828695 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":456,"threads_started":1}
I20260810 06:44:29.830536 23867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:29.806413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40355 (local address 127.10.242.1:39053)
0810 06:44:29.807297 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:29.807360 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:29.809898 (+ 2538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:29.810927 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:29.810941 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:29.811692 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:29.812609 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:29.812643 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.813988 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.813993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:29.816519 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:29.816544 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.817562 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.817571 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:29.817899 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:29.818745 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:29.818778 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:29.821768 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:29.825370 (+ 3602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:29.825383 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:29.825401 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:29.826123 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:29.826538 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:29.826543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:29.826547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:29.826892 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:29.827491 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:29.827499 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:29.827981 (+ 482us) client_negotiation.cc:769] Sending connection context
0810 06:44:29.828338 (+ 357us) client_negotiation.cc:240] Negotiation successful
0810 06:44:29.828937 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":344,"threads_started":1}
I20260810 06:44:29.843423 23824 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:42675
I20260810 06:44:29.843400 24052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:42675 every 8 connection(s)
I20260810 06:44:29.844748 23824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:29.845736 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 23824
I20260810 06:44:29.845940 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:44:29.846820 11208 external_mini_cluster.cc:1428] Setting key 6f4900b4f4da9956078007f0d2867551
I20260810 06:44:29.847223 24053 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40355
I20260810 06:44:29.847458 24053 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:29.849118 24053 heartbeater.cc:511] Master 127.10.242.62:40355 requested a full tablet report, sending...
I20260810 06:44:29.851533 23707 ts_manager.cc:194] Registered new tserver with Master: 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675)
I20260810 06:44:29.853282 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:29.838467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:57933 (local address 127.10.242.62:40355)
0810 06:44:29.838645 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:29.838651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:29.839363 (+ 712us) server_negotiation.cc:407] Connection header received
0810 06:44:29.839549 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:29.839555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:29.839638 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:29.839728 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:29.841069 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.841939 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.843552 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.844026 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.844139 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:29.846565 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:29.846572 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:29.846578 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:29.846622 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:29.850085 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:29.851070 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:29.851076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:29.851081 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:29.851281 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:29.851717 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:29.851722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:29.851725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:29.852059 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:29.852212 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:29.852731 (+ 519us) server_negotiation.cc:299] Negotiation successful
0810 06:44:29.852941 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:44:29.853372 23707 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:39053
I20260810 06:44:29.853317 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:29.838599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40355 (local address 127.10.242.1:57933)
0810 06:44:29.839247 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:29.839267 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:29.839426 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:29.839772 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:29.839781 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:29.840447 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:29.840909 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:29.840928 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.842086 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.842091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:29.843403 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:29.843420 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:29.844225 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:29.844232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:29.844401 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:29.844995 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:29.845018 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:29.846312 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:29.850353 (+ 4041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:29.850359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:29.850364 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:29.850921 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:29.851413 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:29.851419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:29.851422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:29.851585 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:29.852277 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:29.852281 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:29.852464 (+ 183us) client_negotiation.cc:769] Sending connection context
0810 06:44:29.852584 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:44:29.852878 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":377,"threads_started":1}
WARNING: no policy specified for oryx/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab.
Entry for principal oryx/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab.
I20260810 06:44:29.902443 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40355
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab
--principal=oryx/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:30.237327 24057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:30.237672 24057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:30.237815 24057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:30.244935 24057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:30.245052 24057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:30.245174 24057 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:44:30.245285 24057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:30.254832 24057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab
--principal=oryx/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.62:40355
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:30.256089 24057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:30.257503 24057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:30.270498 24063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:30.270423 24065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:30.270462 24070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:30.271421 24057 server_base.cc:1034] running on GCE node
I20260810 06:44:30.271991 24057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:30.272820 24057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:30.274042 24057 hybrid_clock.cc:648] HybridClock initialized: now 1786344270273978 us; error 35 us; skew 500 ppm
I20260810 06:44:30.274351 24057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:30 dist-test-slave-zj8q krb5kdc[23632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344270, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:30.277374 24057 init.cc:377] Logged in from keytab as oryx/127.10.242.2@KRBTEST.COM (short username oryx)
I20260810 06:44:30.278769 24057 webserver.cc:460] Webserver started at http://127.10.242.2:39627/ using document root <none> and password file <none>
I20260810 06:44:30.279397 24057 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:30.279498 24057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:30.279806 24057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:30.282641 24057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "3763b39b7a0a4417ac3e799d229eb150"
format_stamp: "Formatted at 2026-08-10 06:44:30 on dist-test-slave-zj8q"
server_key: "254d6856835602a397c6bdc42d607b61"
server_key_iv: "f3e02e4976ad9cb49cc396ca0c10f14b"
server_key_version: "encryptionkey@0"
I20260810 06:44:30.283388 24057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "3763b39b7a0a4417ac3e799d229eb150"
format_stamp: "Formatted at 2026-08-10 06:44:30 on dist-test-slave-zj8q"
server_key: "254d6856835602a397c6bdc42d607b61"
server_key_iv: "f3e02e4976ad9cb49cc396ca0c10f14b"
server_key_version: "encryptionkey@0"
I20260810 06:44:30.288389 24057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:44:30.291738 24080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:30.292943 24057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:44:30.293082 24057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "3763b39b7a0a4417ac3e799d229eb150"
format_stamp: "Formatted at 2026-08-10 06:44:30 on dist-test-slave-zj8q"
server_key: "254d6856835602a397c6bdc42d607b61"
server_key_iv: "f3e02e4976ad9cb49cc396ca0c10f14b"
server_key_version: "encryptionkey@0"
I20260810 06:44:30.293280 24057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:30.329771 24057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:30.334059 24057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:30.334327 24057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:30.336969 24057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:30.337071 24057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:30.337217 24057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:30.337289 24057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:44:30 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344270, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:30.361146 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:30.337609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:54941 (local address 127.10.242.62:40355)
0810 06:44:30.337777 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:30.337782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:30.339277 (+ 1495us) server_negotiation.cc:407] Connection header received
0810 06:44:30.341250 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:30.341256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:30.341339 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:30.341425 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:30.343649 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.344806 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.347223 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.347939 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.348077 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:30.352675 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:30.352683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:30.352688 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:30.352736 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:30.357922 (+ 5186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:30.358824 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:30.358831 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:30.358835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:30.358998 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:30.359373 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:30.359378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:30.359381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:30.359755 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:30.359908 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:30.360510 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 06:44:30.360752 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:44:30.361980 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:30.338334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40355 (local address 127.10.242.2:54941)
0810 06:44:30.339103 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:30.339163 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:30.340969 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:30.341689 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:30.341697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:30.342485 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:30.343388 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:30.343416 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.345020 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.345024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:30.347064 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:30.347080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.348441 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.348449 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:30.348732 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:30.349443 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:30.349469 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:30.352410 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:30.358050 (+ 5640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:30.358057 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:30.358070 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:30.358668 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:30.359102 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:30.359105 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:30.359107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:30.359274 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:30.359867 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:30.359872 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:30.360177 (+ 305us) client_negotiation.cc:769] Sending connection context
0810 06:44:30.360392 (+ 215us) client_negotiation.cc:240] Negotiation successful
0810 06:44:30.360778 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":324,"threads_started":1}
I20260810 06:44:30.380896 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:30.367476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:60721 (local address 127.10.242.62:40355)
0810 06:44:30.367659 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:30.367664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:30.368688 (+ 1024us) server_negotiation.cc:407] Connection header received
0810 06:44:30.368830 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:30.368835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:30.368914 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:30.369012 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:30.369865 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.371006 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.372751 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.373410 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.373532 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:30.375361 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:30.375366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:30.375369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:30.375399 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:30.378392 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:30.379048 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:30.379053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:30.379057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:30.379177 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:30.379501 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:30.379506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:30.379510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:30.379835 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:30.379966 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:30.380361 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 06:44:30.380576 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:44:30.380985 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:30.367780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40355 (local address 127.10.242.2:60721)
0810 06:44:30.368590 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:30.368605 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:30.368732 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:30.369028 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:30.369031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:30.369458 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:30.369745 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:30.369758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.371158 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.371164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:30.372612 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:30.372624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.373584 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.373589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:30.373727 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:30.374273 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:30.374289 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:30.375158 (+ 869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:30.378511 (+ 3353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:30.378519 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:30.378523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:30.378943 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:30.379277 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:30.379282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:30.379286 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:30.379405 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:30.379947 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:30.379951 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:30.380129 (+ 178us) client_negotiation.cc:769] Sending connection context
0810 06:44:30.380250 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:44:30.380550 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":550,"threads_started":1}
I20260810 06:44:30.393409 24057 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:35097
I20260810 06:44:30.393430 24266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:35097 every 8 connection(s)
I20260810 06:44:30.394580 24057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:30.396515 24267 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40355
I20260810 06:44:30.396741 24267 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:30.397360 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 24057
I20260810 06:44:30.397377 24267 heartbeater.cc:511] Master 127.10.242.62:40355 requested a full tablet report, sending...
I20260810 06:44:30.397511 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:44:30.397975 11208 external_mini_cluster.cc:1428] Setting key 0f67427ca97c2889bdec97ee074a514b
I20260810 06:44:30.398940 23707 ts_manager.cc:194] Registered new tserver with Master: 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097)
I20260810 06:44:30.400193 23707 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:54941
WARNING: no policy specified for oryx/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab.
Entry for principal oryx/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab.
I20260810 06:44:30.447347 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40355
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab
--principal=oryx/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:30.762563 24273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:30.762828 24273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:30.762960 24273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:30.770134 24273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:30.770263 24273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:30.770388 24273 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:44:30.770491 24273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:30.779838 24273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab
--principal=oryx/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.242.62:40355
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:30.781030 24273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:30.782356 24273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:30.794996 24280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:30.795053 24279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:30.795101 24286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:30.796052 24273 server_base.cc:1034] running on GCE node
I20260810 06:44:30.796617 24273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:30.797456 24273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:30.798656 24273 hybrid_clock.cc:648] HybridClock initialized: now 1786344270798576 us; error 56 us; skew 500 ppm
I20260810 06:44:30.798941 24273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:30 dist-test-slave-zj8q krb5kdc[23632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344270, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:30.801781 24273 init.cc:377] Logged in from keytab as oryx/127.10.242.3@KRBTEST.COM (short username oryx)
I20260810 06:44:30.803095 24273 webserver.cc:460] Webserver started at http://127.10.242.3:46581/ using document root <none> and password file <none>
I20260810 06:44:30.803740 24273 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:30.803840 24273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:30.804154 24273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:30.806999 24273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "c59ad7fe4f7847a88023208774885744"
format_stamp: "Formatted at 2026-08-10 06:44:30 on dist-test-slave-zj8q"
server_key: "e6b0aac497c570ffedb08e2d89c8d2a7"
server_key_iv: "3f49f10ce3adc14ce87149a68f7d85d8"
server_key_version: "encryptionkey@0"
I20260810 06:44:30.807770 24273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "c59ad7fe4f7847a88023208774885744"
format_stamp: "Formatted at 2026-08-10 06:44:30 on dist-test-slave-zj8q"
server_key: "e6b0aac497c570ffedb08e2d89c8d2a7"
server_key_iv: "3f49f10ce3adc14ce87149a68f7d85d8"
server_key_version: "encryptionkey@0"
I20260810 06:44:30.812539 24273 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 06:44:30.815747 24295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:30.816923 24273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:44:30.817073 24273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "c59ad7fe4f7847a88023208774885744"
format_stamp: "Formatted at 2026-08-10 06:44:30 on dist-test-slave-zj8q"
server_key: "e6b0aac497c570ffedb08e2d89c8d2a7"
server_key_iv: "3f49f10ce3adc14ce87149a68f7d85d8"
server_key_version: "encryptionkey@0"
I20260810 06:44:30.817281 24273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:30.841760 24273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:30.846575 24273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:30.846886 24273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:30.849978 24273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:30.850071 24273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:30.850267 24273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:30.850346 24273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:30.856492 24053 heartbeater.cc:503] Master 127.10.242.62:40355 was elected leader, sending a full tablet report...
Aug 10 06:44:30 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344270, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:30.870930 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:30.850061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:56437 (local address 127.10.242.62:40355)
0810 06:44:30.850286 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:30.850293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:30.851543 (+ 1250us) server_negotiation.cc:407] Connection header received
0810 06:44:30.853506 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:30.853512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:30.853589 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:30.853675 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:30.855641 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.856765 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.859190 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.859722 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.859833 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:30.864278 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:30.864284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:30.864290 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:30.864329 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:30.867286 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:30.868147 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:30.868152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:30.868155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:30.868355 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:30.868927 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:30.868930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:30.868932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:30.869198 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:30.869344 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:30.870354 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0810 06:44:30.870583 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:44:30.871810 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:30.850563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40355 (local address 127.10.242.3:56437)
0810 06:44:30.851377 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:30.851426 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:30.853250 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:30.853909 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:30.853916 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:30.854580 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:30.855373 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:30.855395 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.856955 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.856961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:30.859018 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:30.859036 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.859967 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.859975 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:30.860187 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:30.860797 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:30.860826 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:30.864037 (+ 3211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:30.867409 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:30.867422 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:30.867436 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:30.868024 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:30.868487 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:30.868493 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:30.868498 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:30.868787 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:30.869430 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:30.869439 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:30.869851 (+ 412us) client_negotiation.cc:769] Sending connection context
0810 06:44:30.870215 (+ 364us) client_negotiation.cc:240] Negotiation successful
0810 06:44:30.870709 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":330,"threads_started":1}
I20260810 06:44:30.895284 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:30.880733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:49069 (local address 127.10.242.62:40355)
0810 06:44:30.880876 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:30.880882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:30.881949 (+ 1067us) server_negotiation.cc:407] Connection header received
0810 06:44:30.882109 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:30.882112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:30.882229 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:30.882329 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:30.883800 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.884682 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.886607 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.887124 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.887216 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:30.889092 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:30.889100 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:30.889105 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:30.889148 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:30.892396 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:30.893145 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:30.893151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:30.893155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:30.893308 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:30.893652 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:30.893657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:30.893661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:30.894042 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:30.894230 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:30.894718 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 06:44:30.894930 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:44:30.895393 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:30.880715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40355 (local address 127.10.242.3:49069)
0810 06:44:30.881840 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:30.881860 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:30.882006 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:30.882390 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:30.882395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:30.883139 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:30.883619 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:30.883654 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.884847 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.884854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:30.886413 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:30.886426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.887335 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.887343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:30.887528 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:30.888167 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:30.888189 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:30.888885 (+ 696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:30.892540 (+ 3655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:30.892546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:30.892550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:30.893022 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:30.893411 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:30.893414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:30.893416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:30.893538 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:30.894209 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:30.894219 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:30.894436 (+ 217us) client_negotiation.cc:769] Sending connection context
0810 06:44:30.894591 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 06:44:30.894933 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"thread_start_us":776,"threads_started":1}
I20260810 06:44:30.907351 24273 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:40777
I20260810 06:44:30.907375 24489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:40777 every 8 connection(s)
I20260810 06:44:30.908535 24273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:30.910384 24490 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40355
I20260810 06:44:30.910589 24490 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:30.911180 24490 heartbeater.cc:511] Master 127.10.242.62:40355 requested a full tablet report, sending...
I20260810 06:44:30.912377 23707 ts_manager.cc:194] Registered new tserver with Master: c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777)
I20260810 06:44:30.913147 23707 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:56437
I20260810 06:44:30.917435 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 24273
I20260810 06:44:30.917587 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:44:30.918088 11208 external_mini_cluster.cc:1428] Setting key cc9a80eebdef5ad5c79aa407a3e2f88d
I20260810 06:44:30.920903 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:44:30 dist-test-slave-zj8q krb5kdc[23632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344270, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:44:30 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:30 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:31.402354 24267 heartbeater.cc:503] Master 127.10.242.62:40355 was elected leader, sending a full tablet report...
I20260810 06:44:31.915328 24490 heartbeater.cc:503] Master 127.10.242.62:40355 was elected leader, sending a full tablet report...
W20260810 06:44:31.984481 24501 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.014s user 0.010s sys 0.007s
W20260810 06:44:31.985581 23866 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:30.964628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.10.242.62:40355)
0810 06:44:30.964852 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:30.964858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:30.965165 (+ 307us) server_negotiation.cc:407] Connection header received
0810 06:44:30.965323 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:30.965327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:30.965408 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:30.965492 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:30.966845 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.967570 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.968854 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:30.969325 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:30.969418 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:31.985339 (+1015921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53348: BlockingRecv error: recv got EOF from 127.0.0.1:53348 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 10 06:44:31 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344270, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:32.006250 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:31.989921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.10.242.62:40355)
0810 06:44:31.990062 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:31.990066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:31.990449 (+ 383us) server_negotiation.cc:407] Connection header received
0810 06:44:31.990599 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:31.990602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:31.990712 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:31.990796 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:31.992213 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:31.992980 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:31.994317 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:31.994777 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:31.994874 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.001634 (+ 6760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:32.001639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:32.001642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:32.001686 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:32.004191 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:32.004775 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:32.004778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:32.004781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:32.004896 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:32.005177 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:32.005180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:32.005181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:32.005435 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:32.005689 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.005833 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.005998 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:44:32.009927 23707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:44:32.018363 23707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:44:32.041020 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.030552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47814 (local address 127.10.242.2:35097)
0810 06:44:32.031094 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.031103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.033579 (+ 2476us) server_negotiation.cc:407] Connection header received
0810 06:44:32.033704 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.033710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.033896 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.034111 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:32.034770 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.035758 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.037955 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.039321 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.039822 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.040031 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.040562 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.040735 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":250,"threads_started":1}
I20260810 06:44:32.041060 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.030345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:35097 (local address 127.0.0.1:47814)
0810 06:44:32.032386 (+ 2041us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.032427 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.032608 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.034138 (+ 1530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.034146 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.034209 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:32.034600 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.034611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.035914 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.035920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.037763 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.037777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.039540 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.039547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.040316 (+ 769us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.040383 (+ 67us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.040500 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.040697 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1779,"mutex_wait_us":505,"thread_start_us":8366,"threads_started":1}
I20260810 06:44:32.045198 24157 tablet_service.cc:1467] Processing CreateTablet for tablet 42e0b8e279e44dd5875c4f323c79173e (DEFAULT_TABLE table=test-table [id=905a6bab42aa4169b46e65923cc6a389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:32.046033 24524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.030621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:40777 (local address 127.0.0.1:43182)
0810 06:44:32.038602 (+ 7981us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.038624 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.038846 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.039713 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.039717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.039742 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:32.040228 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.040242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.042123 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.042129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.043744 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.043758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.045284 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.045290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.045445 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.045470 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.045559 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.045677 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7858,"mutex_wait_us":6,"thread_start_us":7717,"threads_started":1}
I20260810 06:44:32.046062 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.031227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.10.242.3:40777)
0810 06:44:32.031821 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.031830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.038725 (+ 6895us) server_negotiation.cc:407] Connection header received
0810 06:44:32.039042 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.039049 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.039279 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.039472 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:32.040462 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.041981 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.043926 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.045082 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.045204 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.045399 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.045618 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.045793 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":290,"threads_started":1}
I20260810 06:44:32.047067 24157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e0b8e279e44dd5875c4f323c79173e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.047238 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.031650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:42675 (local address 127.0.0.1:38638)
0810 06:44:32.038963 (+ 7313us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.038979 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.039098 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.039900 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.039904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.039930 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:32.040380 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.040393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.042248 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.042252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.044423 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.044438 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.046508 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.046514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.046682 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.046713 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.046796 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.046907 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7195,"mutex_wait_us":17,"thread_start_us":710,"threads_started":1}
I20260810 06:44:32.047945 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.032152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.10.242.1:42675)
0810 06:44:32.033145 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.033154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.039084 (+ 5930us) server_negotiation.cc:407] Connection header received
0810 06:44:32.039278 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.039285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.039500 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.039687 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:32.040592 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.042051 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.044600 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.045994 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.046101 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.046318 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.047513 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.047683 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":1180,"threads_started":1}
I20260810 06:44:32.049300 24384 tablet_service.cc:1467] Processing CreateTablet for tablet 42e0b8e279e44dd5875c4f323c79173e (DEFAULT_TABLE table=test-table [id=905a6bab42aa4169b46e65923cc6a389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:32.050453 24384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e0b8e279e44dd5875c4f323c79173e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.050715 23936 tablet_service.cc:1467] Processing CreateTablet for tablet 42e0b8e279e44dd5875c4f323c79173e (DEFAULT_TABLE table=test-table [id=905a6bab42aa4169b46e65923cc6a389]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:32.051733 23936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42e0b8e279e44dd5875c4f323c79173e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.061689 24530 tablet_bootstrap.cc:492] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: Bootstrap starting.
I20260810 06:44:32.063827 24532 tablet_bootstrap.cc:492] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: Bootstrap starting.
I20260810 06:44:32.065474 24530 tablet_bootstrap.cc:654] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:32.066813 24530 log.cc:824] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:32.067126 24533 tablet_bootstrap.cc:492] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: Bootstrap starting.
I20260810 06:44:32.067257 24532 tablet_bootstrap.cc:654] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:32.068818 24532 log.cc:824] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:32.070312 24533 tablet_bootstrap.cc:654] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:32.070495 24530 tablet_bootstrap.cc:492] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: No bootstrap required, opened a new log
I20260810 06:44:32.070780 24530 ts_tablet_manager.cc:1397] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 06:44:32.071434 24533 log.cc:824] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:32.072045 24532 tablet_bootstrap.cc:492] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: No bootstrap required, opened a new log
I20260810 06:44:32.072309 24532 ts_tablet_manager.cc:1397] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:44:32.074064 24533 tablet_bootstrap.cc:492] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: No bootstrap required, opened a new log
I20260810 06:44:32.074326 24533 ts_tablet_manager.cc:1397] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.005s
I20260810 06:44:32.080793 24533 raft_consensus.cc:348] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } }
I20260810 06:44:32.081152 24533 raft_consensus.cc:374] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:32.080919 24530 raft_consensus.cc:348] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } }
I20260810 06:44:32.081254 24533 raft_consensus.cc:729] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3763b39b7a0a4417ac3e799d229eb150, State: Initialized, Role: FOLLOWER
I20260810 06:44:32.081310 24530 raft_consensus.cc:374] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:32.081400 24530 raft_consensus.cc:729] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8adf2f1d144af59fa1868bd576e51d, State: Initialized, Role: FOLLOWER
I20260810 06:44:32.081910 24533 consensus_queue.cc:260] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } }
I20260810 06:44:32.082048 24530 consensus_queue.cc:260] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } }
I20260810 06:44:32.082267 24532 raft_consensus.cc:348] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } }
I20260810 06:44:32.082670 24532 raft_consensus.cc:374] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:32.082773 24532 raft_consensus.cc:729] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad7fe4f7847a88023208774885744, State: Initialized, Role: FOLLOWER
I20260810 06:44:32.083313 24532 consensus_queue.cc:260] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } }
I20260810 06:44:32.084040 24533 ts_tablet_manager.cc:1428] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
I20260810 06:44:32.085747 24532 ts_tablet_manager.cc:1428] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260810 06:44:32.087587 24530 ts_tablet_manager.cc:1428] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: Time spent starting tablet: real 0.017s user 0.013s sys 0.000s
W20260810 06:44:32.103773 24054 tablet.cc:2369] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:44:32.147269 24269 tablet.cc:2369] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:44:32.160696 24492 tablet.cc:2369] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:44:32.244340 24544 raft_consensus.cc:482] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:32.244638 24544 raft_consensus.cc:504] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } }
I20260810 06:44:32.246528 24544 leader_election.cc:288] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097), c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777)
I20260810 06:44:32.254701 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.246947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:37947 (local address 127.10.242.2:35097)
0810 06:44:32.247161 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.247167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.248168 (+ 1001us) server_negotiation.cc:407] Connection header received
0810 06:44:32.248312 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.248318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.248383 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.248447 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:32.249055 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.250081 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.252442 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.253661 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.253823 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.253897 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.254314 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.254454 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:44:32.254706 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.247056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:58549 (local address 127.10.242.3:40777)
0810 06:44:32.247240 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.247245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.247822 (+ 577us) server_negotiation.cc:407] Connection header received
0810 06:44:32.248035 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.248041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.248107 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.248199 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:32.248796 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.249852 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.252438 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.253660 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.253829 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.253897 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.254314 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.254454 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:44:32.254901 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.247008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:35097 (local address 127.10.242.1:37947)
0810 06:44:32.248050 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.248065 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.248194 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.248499 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.248502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.248524 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:32.248898 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.248908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.250214 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.250219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.252271 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.252286 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.253930 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.253936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.254144 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.254190 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.254263 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.254435 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"mutex_wait_us":100,"thread_start_us":761,"threads_started":1}
I20260810 06:44:32.254922 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.246979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:40777 (local address 127.10.242.1:58549)
0810 06:44:32.247702 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.247718 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.247886 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.248242 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.248247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.248273 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:32.248638 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.248651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.249979 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.249983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.252270 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.252286 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.253900 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.253907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.254108 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.254146 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.254263 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.254435 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":47,"thread_start_us":390,"threads_started":1}
I20260810 06:44:32.255982 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e0b8e279e44dd5875c4f323c79173e" candidate_uuid: "6a8adf2f1d144af59fa1868bd576e51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad7fe4f7847a88023208774885744" is_pre_election: true
I20260810 06:44:32.255982 24195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e0b8e279e44dd5875c4f323c79173e" candidate_uuid: "6a8adf2f1d144af59fa1868bd576e51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3763b39b7a0a4417ac3e799d229eb150" is_pre_election: true
I20260810 06:44:32.256563 24414 raft_consensus.cc:2454] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8adf2f1d144af59fa1868bd576e51d in term 0.
I20260810 06:44:32.256563 24195 raft_consensus.cc:2454] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8adf2f1d144af59fa1868bd576e51d in term 0.
I20260810 06:44:32.257254 23858 leader_election.cc:302] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3763b39b7a0a4417ac3e799d229eb150, 6a8adf2f1d144af59fa1868bd576e51d; no voters:
I20260810 06:44:32.257665 24544 raft_consensus.cc:2781] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:44:32.257807 24544 raft_consensus.cc:482] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:32.257895 24544 raft_consensus.cc:3037] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:32.260123 24544 raft_consensus.cc:504] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } }
I20260810 06:44:32.260689 24544 leader_election.cc:288] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [CANDIDATE]: Term 1 election: Requested vote from peers 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097), c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777)
I20260810 06:44:32.261067 24195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e0b8e279e44dd5875c4f323c79173e" candidate_uuid: "6a8adf2f1d144af59fa1868bd576e51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3763b39b7a0a4417ac3e799d229eb150"
I20260810 06:44:32.261140 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42e0b8e279e44dd5875c4f323c79173e" candidate_uuid: "6a8adf2f1d144af59fa1868bd576e51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad7fe4f7847a88023208774885744"
I20260810 06:44:32.261248 24195 raft_consensus.cc:3037] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:32.261329 24414 raft_consensus.cc:3037] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:32.263409 24195 raft_consensus.cc:2454] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8adf2f1d144af59fa1868bd576e51d in term 1.
I20260810 06:44:32.263460 24414 raft_consensus.cc:2454] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8adf2f1d144af59fa1868bd576e51d in term 1.
I20260810 06:44:32.263770 23858 leader_election.cc:302] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3763b39b7a0a4417ac3e799d229eb150, 6a8adf2f1d144af59fa1868bd576e51d; no voters:
I20260810 06:44:32.264025 24544 raft_consensus.cc:2781] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:32.264406 24544 raft_consensus.cc:686] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 1 LEADER]: Becoming Leader. State: Replica: 6a8adf2f1d144af59fa1868bd576e51d, State: Running, Role: LEADER
I20260810 06:44:32.264876 24544 consensus_queue.cc:237] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } }
I20260810 06:44:32.269155 23704 catalog_manager.cc:5495] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1). New cstate: current_term: 1 leader_uuid: "6a8adf2f1d144af59fa1868bd576e51d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } health_report { overall_health: HEALTHY } } }
I20260810 06:44:32.340497 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.335182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.10.242.1:42675)
0810 06:44:32.335406 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.335413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.335499 (+ 86us) server_negotiation.cc:407] Connection header received
0810 06:44:32.335748 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.335752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.335844 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.335976 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:32.336338 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.337042 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.338421 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.338847 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.338955 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.339388 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:32.339933 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:32.340064 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.340155 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.340238 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:44:32.348498 24414 raft_consensus.cc:1261] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Refusing update from remote peer 6a8adf2f1d144af59fa1868bd576e51d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:32.348500 24195 raft_consensus.cc:1261] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Refusing update from remote peer 6a8adf2f1d144af59fa1868bd576e51d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:32.349630 24550 consensus_queue.cc:1035] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:32.350041 24544 consensus_queue.cc:1035] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:32.363623 24553 mvcc.cc:205] Tried to move back new op lower bound from 7316866139529375744 to 7316866139200327680. Current Snapshot: MvccSnapshot[applied={T|T < 7316866139529375744}]
I20260810 06:44:32.367717 24557 mvcc.cc:205] Tried to move back new op lower bound from 7316866139529375744 to 7316866139200327680. Current Snapshot: MvccSnapshot[applied={T|T < 7316866139529375744}]
I20260810 06:44:32.376228 23704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:32.376482 23704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:32.379806 23704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d: Sending DeleteTablet for 3 replicas of tablet 42e0b8e279e44dd5875c4f323c79173e
I20260810 06:44:32.380872 24157 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e0b8e279e44dd5875c4f323c79173e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:32 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:47814
I20260810 06:44:32.380905 24384 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e0b8e279e44dd5875c4f323c79173e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:32 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:43182
I20260810 06:44:32.381188 23936 tablet_service.cc:1514] Processing DeleteTablet for tablet 42e0b8e279e44dd5875c4f323c79173e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:32 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:38638
I20260810 06:44:32.383908 23704 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:44:32.384536 23704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:44:32.392581 24383 tablet_service.cc:1467] Processing CreateTablet for tablet 214c32bb1a1a424483d67d177f8ac498 (DEFAULT_TABLE table=test-table [id=d138a3032c3347ac959907ab0bc58174]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:32.393252 24383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 214c32bb1a1a424483d67d177f8ac498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.396070 24571 tablet_replica.cc:330] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: stopping tablet replica
I20260810 06:44:32.396535 24571 raft_consensus.cc:2229] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:44:32.397210 24571 raft_consensus.cc:2258] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:32.399037 24156 tablet_service.cc:1467] Processing CreateTablet for tablet 214c32bb1a1a424483d67d177f8ac498 (DEFAULT_TABLE table=test-table [id=d138a3032c3347ac959907ab0bc58174]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:32.399590 24156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 214c32bb1a1a424483d67d177f8ac498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.401196 24571 ts_tablet_manager.cc:1905] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:32.407613 24570 tablet_replica.cc:330] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: stopping tablet replica
I20260810 06:44:32.408064 24570 raft_consensus.cc:2229] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:32.408545 24570 raft_consensus.cc:2258] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:32.408507 23936 tablet_service.cc:1467] Processing CreateTablet for tablet 214c32bb1a1a424483d67d177f8ac498 (DEFAULT_TABLE table=test-table [id=d138a3032c3347ac959907ab0bc58174]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:32.409246 23936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 214c32bb1a1a424483d67d177f8ac498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.410632 24533 tablet_bootstrap.cc:492] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: Bootstrap starting.
I20260810 06:44:32.410722 24570 ts_tablet_manager.cc:1905] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:32.412324 24533 tablet_bootstrap.cc:654] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:32.417243 24570 ts_tablet_manager.cc:1918] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:32.417394 24570 log.cc:1196] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/42e0b8e279e44dd5875c4f323c79173e
I20260810 06:44:32.417901 24570 ts_tablet_manager.cc:1939] T 42e0b8e279e44dd5875c4f323c79173e P c59ad7fe4f7847a88023208774885744: Deleting consensus metadata
I20260810 06:44:32.418286 24532 tablet_bootstrap.cc:492] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: Bootstrap starting.
I20260810 06:44:32.419759 24532 tablet_bootstrap.cc:654] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:32.420488 24571 ts_tablet_manager.cc:1918] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:32.420672 24571 log.cc:1196] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/42e0b8e279e44dd5875c4f323c79173e
I20260810 06:44:32.421190 24571 ts_tablet_manager.cc:1939] T 42e0b8e279e44dd5875c4f323c79173e P 6a8adf2f1d144af59fa1868bd576e51d: Deleting consensus metadata
I20260810 06:44:32.422994 23685 catalog_manager.cc:4841] TS 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675): tablet 42e0b8e279e44dd5875c4f323c79173e (table test-table [id=905a6bab42aa4169b46e65923cc6a389]) successfully deleted
I20260810 06:44:32.424117 24569 tablet_replica.cc:330] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: stopping tablet replica
I20260810 06:44:32.424671 24569 raft_consensus.cc:2229] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:32.424715 23687 catalog_manager.cc:4841] TS c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777): tablet 42e0b8e279e44dd5875c4f323c79173e (table test-table [id=905a6bab42aa4169b46e65923cc6a389]) successfully deleted
I20260810 06:44:32.425240 24569 raft_consensus.cc:2258] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:32.430436 24569 ts_tablet_manager.cc:1905] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:32.431331 24530 tablet_bootstrap.cc:492] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: Bootstrap starting.
I20260810 06:44:32.432992 24530 tablet_bootstrap.cc:654] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:32.438283 24569 ts_tablet_manager.cc:1918] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:32.438494 24569 log.cc:1196] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/42e0b8e279e44dd5875c4f323c79173e
I20260810 06:44:32.438907 24532 tablet_bootstrap.cc:492] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: No bootstrap required, opened a new log
I20260810 06:44:32.439061 24532 ts_tablet_manager.cc:1397] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.007s
I20260810 06:44:32.439110 24569 ts_tablet_manager.cc:1939] T 42e0b8e279e44dd5875c4f323c79173e P 3763b39b7a0a4417ac3e799d229eb150: Deleting consensus metadata
I20260810 06:44:32.439872 24532 raft_consensus.cc:348] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.440112 24532 raft_consensus.cc:374] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:32.440208 24532 raft_consensus.cc:729] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad7fe4f7847a88023208774885744, State: Initialized, Role: FOLLOWER
I20260810 06:44:32.440485 24532 consensus_queue.cc:260] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.441062 24532 ts_tablet_manager.cc:1428] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 06:44:32.441151 23686 catalog_manager.cc:4841] TS 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097): tablet 42e0b8e279e44dd5875c4f323c79173e (table test-table [id=905a6bab42aa4169b46e65923cc6a389]) successfully deleted
I20260810 06:44:32.442116 24533 tablet_bootstrap.cc:492] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: No bootstrap required, opened a new log
I20260810 06:44:32.442407 24533 ts_tablet_manager.cc:1397] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.009s
I20260810 06:44:32.442675 24530 tablet_bootstrap.cc:492] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: No bootstrap required, opened a new log
I20260810 06:44:32.442857 24530 ts_tablet_manager.cc:1397] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260810 06:44:32.443318 24533 raft_consensus.cc:348] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.443588 24533 raft_consensus.cc:374] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:32.443717 24533 raft_consensus.cc:729] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3763b39b7a0a4417ac3e799d229eb150, State: Initialized, Role: FOLLOWER
I20260810 06:44:32.443742 24530 raft_consensus.cc:348] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.443970 24530 raft_consensus.cc:374] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:32.444085 24530 raft_consensus.cc:729] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8adf2f1d144af59fa1868bd576e51d, State: Initialized, Role: FOLLOWER
I20260810 06:44:32.444098 24533 consensus_queue.cc:260] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.444427 24530 consensus_queue.cc:260] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.444896 24533 ts_tablet_manager.cc:1428] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:44:32.445511 24530 ts_tablet_manager.cc:1428] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:32.522496 24543 raft_consensus.cc:482] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:32.522713 24543 raft_consensus.cc:504] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.524284 24543 leader_election.cc:288] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675), c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777)
I20260810 06:44:32.532585 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.524824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:35517 (local address 127.10.242.3:40777)
0810 06:44:32.525022 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.525030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.526360 (+ 1330us) server_negotiation.cc:407] Connection header received
0810 06:44:32.526540 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.526545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.526621 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.526709 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:32.527386 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.528534 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.530773 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.531554 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.531708 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.531783 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.532157 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.532309 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:44:32.532750 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.525270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:50497 (local address 127.10.242.1:42675)
0810 06:44:32.525453 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.525458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.526090 (+ 632us) server_negotiation.cc:407] Connection header received
0810 06:44:32.526375 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.526381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.526481 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.526570 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:32.527296 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.528112 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.530524 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.531766 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.531930 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.532002 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.532325 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.532463 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:44:32.532601 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.524662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:40777 (local address 127.10.242.2:35517)
0810 06:44:32.526267 (+ 1605us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.526284 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.526409 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.526755 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.526760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.526784 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:32.527169 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.527178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.528676 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.528681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.530637 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.530647 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.531763 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.531769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.531959 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.531996 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.532095 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.532220 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1435,"mutex_wait_us":7,"thread_start_us":953,"threads_started":1}
I20260810 06:44:32.533432 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.524519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:42675 (local address 127.10.242.2:50497)
0810 06:44:32.525977 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.525999 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.526223 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.526608 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.526613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.526640 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:32.527063 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.527080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.528244 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.528250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.530353 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.530368 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.531973 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.531979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.532149 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.532180 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.533027 (+ 847us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.533162 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1309,"mutex_wait_us":6,"thread_start_us":1419,"threads_started":1}
I20260810 06:44:32.533635 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "214c32bb1a1a424483d67d177f8ac498" candidate_uuid: "3763b39b7a0a4417ac3e799d229eb150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad7fe4f7847a88023208774885744" is_pre_election: true
I20260810 06:44:32.533914 24414 raft_consensus.cc:2454] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3763b39b7a0a4417ac3e799d229eb150 in term 0.
I20260810 06:44:32.534428 24088 leader_election.cc:302] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3763b39b7a0a4417ac3e799d229eb150, c59ad7fe4f7847a88023208774885744; no voters:
I20260810 06:44:32.534315 23968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "214c32bb1a1a424483d67d177f8ac498" candidate_uuid: "3763b39b7a0a4417ac3e799d229eb150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8adf2f1d144af59fa1868bd576e51d" is_pre_election: true
I20260810 06:44:32.534745 24543 raft_consensus.cc:2781] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:44:32.534757 23968 raft_consensus.cc:2454] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3763b39b7a0a4417ac3e799d229eb150 in term 0.
I20260810 06:44:32.534857 24543 raft_consensus.cc:482] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:32.534940 24543 raft_consensus.cc:3037] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:32.537391 24543 raft_consensus.cc:504] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.537912 24543 leader_election.cc:288] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [CANDIDATE]: Term 1 election: Requested vote from peers 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675), c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777)
I20260810 06:44:32.538398 23968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "214c32bb1a1a424483d67d177f8ac498" candidate_uuid: "3763b39b7a0a4417ac3e799d229eb150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8adf2f1d144af59fa1868bd576e51d"
I20260810 06:44:32.538403 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "214c32bb1a1a424483d67d177f8ac498" candidate_uuid: "3763b39b7a0a4417ac3e799d229eb150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad7fe4f7847a88023208774885744"
I20260810 06:44:32.538599 24414 raft_consensus.cc:3037] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:32.538602 23968 raft_consensus.cc:3037] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:32.540494 24414 raft_consensus.cc:2454] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3763b39b7a0a4417ac3e799d229eb150 in term 1.
I20260810 06:44:32.540836 24088 leader_election.cc:302] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3763b39b7a0a4417ac3e799d229eb150, c59ad7fe4f7847a88023208774885744; no voters:
I20260810 06:44:32.541072 24543 raft_consensus.cc:2781] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:32.541698 24543 raft_consensus.cc:686] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 1 LEADER]: Becoming Leader. State: Replica: 3763b39b7a0a4417ac3e799d229eb150, State: Running, Role: LEADER
I20260810 06:44:32.541859 23968 raft_consensus.cc:2454] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3763b39b7a0a4417ac3e799d229eb150 in term 1.
I20260810 06:44:32.542100 24543 consensus_queue.cc:237] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.545483 23707 catalog_manager.cc:5495] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2). New cstate: current_term: 1 leader_uuid: "3763b39b7a0a4417ac3e799d229eb150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:44:32 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344269, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.62@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:32.606380 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.592330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40355 (local address 127.0.0.1:53354)
0810 06:44:32.593459 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.593476 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.593599 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.593981 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.593984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.594661 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:32.594861 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.594871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.596382 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.596385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.597403 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.597416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.598193 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.598199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.598361 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.598964 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:32.598981 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:32.601249 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:32.603977 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:32.603990 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:32.603999 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:32.604379 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:32.604705 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:32.604710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:32.604714 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:32.604846 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:32.605347 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:32.605354 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:32.605507 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.605678 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.605924 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"thread_start_us":857,"threads_started":1}
I20260810 06:44:32.606428 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.592418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53354 (local address 127.10.242.62:40355)
0810 06:44:32.593315 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.593321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.593562 (+ 241us) server_negotiation.cc:407] Connection header received
0810 06:44:32.593715 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.593721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.593816 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.593939 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:32.595031 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.596249 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.597528 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.597997 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.598103 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.601422 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:32.601427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:32.601431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:32.601458 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:32.603854 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:32.604483 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:32.604488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:32.604490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:32.604606 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:32.604941 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:32.604946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:32.604950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:32.605226 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:32.605373 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.605776 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.606049 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":709,"thread_start_us":555,"threads_started":1}
I20260810 06:44:32.611035 23707 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:53354:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:44:32.611972 23707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:44:32.618461 23936 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae11cee4f5f45019fd76e7ce9399dda (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0a8e30bc51eb4f58997fdf8002b59e78]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:44:32.618876 24383 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae11cee4f5f45019fd76e7ce9399dda (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0a8e30bc51eb4f58997fdf8002b59e78]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:44:32.618860 24157 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae11cee4f5f45019fd76e7ce9399dda (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0a8e30bc51eb4f58997fdf8002b59e78]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:44:32.619073 23936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae11cee4f5f45019fd76e7ce9399dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.619367 24383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae11cee4f5f45019fd76e7ce9399dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.619385 24157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae11cee4f5f45019fd76e7ce9399dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.626348 24532 tablet_bootstrap.cc:492] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: Bootstrap starting.
I20260810 06:44:32.626787 24530 tablet_bootstrap.cc:492] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: Bootstrap starting.
I20260810 06:44:32.627885 24532 tablet_bootstrap.cc:654] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:32.628479 24530 tablet_bootstrap.cc:654] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:32.629120 24533 tablet_bootstrap.cc:492] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: Bootstrap starting.
I20260810 06:44:32.630776 24533 tablet_bootstrap.cc:654] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:32.631109 24532 tablet_bootstrap.cc:492] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: No bootstrap required, opened a new log
I20260810 06:44:32.631280 24532 ts_tablet_manager.cc:1397] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:44:32.633042 24532 raft_consensus.cc:348] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.633385 24532 raft_consensus.cc:374] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:32.633512 24532 raft_consensus.cc:729] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad7fe4f7847a88023208774885744, State: Initialized, Role: FOLLOWER
I20260810 06:44:32.634107 24532 consensus_queue.cc:260] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.634644 24546 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.635001 24546 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:44:32.638720 24532 ts_tablet_manager.cc:1428] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260810 06:44:32.639539 24533 tablet_bootstrap.cc:492] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: No bootstrap required, opened a new log
I20260810 06:44:32.639704 24533 ts_tablet_manager.cc:1397] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260810 06:44:32.640290 24530 tablet_bootstrap.cc:492] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: No bootstrap required, opened a new log
I20260810 06:44:32.640519 24530 ts_tablet_manager.cc:1397] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260810 06:44:32.640427 24533 raft_consensus.cc:348] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.640662 24533 raft_consensus.cc:374] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:32.640745 24533 raft_consensus.cc:729] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3763b39b7a0a4417ac3e799d229eb150, State: Initialized, Role: FOLLOWER
I20260810 06:44:32.640985 24533 consensus_queue.cc:260] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.641446 24543 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.641759 24543 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:44:32.641618 24533 ts_tablet_manager.cc:1428] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:44:32.642261 24530 raft_consensus.cc:348] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.642516 24530 raft_consensus.cc:374] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:32.642611 24530 raft_consensus.cc:729] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8adf2f1d144af59fa1868bd576e51d, State: Initialized, Role: FOLLOWER
I20260810 06:44:32.642858 24530 consensus_queue.cc:260] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.643241 24544 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.643502 24544 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:44:32.644407 24530 ts_tablet_manager.cc:1428] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 06:44:32.682230 24543 raft_consensus.cc:482] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:32.682404 24543 raft_consensus.cc:504] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.682958 24543 leader_election.cc:288] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675), c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777)
I20260810 06:44:32.683527 23968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae11cee4f5f45019fd76e7ce9399dda" candidate_uuid: "3763b39b7a0a4417ac3e799d229eb150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8adf2f1d144af59fa1868bd576e51d" is_pre_election: true
I20260810 06:44:32.683497 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae11cee4f5f45019fd76e7ce9399dda" candidate_uuid: "3763b39b7a0a4417ac3e799d229eb150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad7fe4f7847a88023208774885744" is_pre_election: true
I20260810 06:44:32.683777 24414 raft_consensus.cc:2454] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3763b39b7a0a4417ac3e799d229eb150 in term 0.
I20260810 06:44:32.683784 23968 raft_consensus.cc:2454] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3763b39b7a0a4417ac3e799d229eb150 in term 0.
I20260810 06:44:32.684217 24088 leader_election.cc:302] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3763b39b7a0a4417ac3e799d229eb150, c59ad7fe4f7847a88023208774885744; no voters:
I20260810 06:44:32.684504 24543 raft_consensus.cc:2781] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:44:32.684592 24543 raft_consensus.cc:482] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:32.684706 24543 raft_consensus.cc:3037] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:32.686789 24543 raft_consensus.cc:504] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.687340 24543 leader_election.cc:288] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [CANDIDATE]: Term 1 election: Requested vote from peers 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675), c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777)
I20260810 06:44:32.687811 23968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae11cee4f5f45019fd76e7ce9399dda" candidate_uuid: "3763b39b7a0a4417ac3e799d229eb150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8adf2f1d144af59fa1868bd576e51d"
I20260810 06:44:32.687821 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae11cee4f5f45019fd76e7ce9399dda" candidate_uuid: "3763b39b7a0a4417ac3e799d229eb150" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad7fe4f7847a88023208774885744"
I20260810 06:44:32.688048 23968 raft_consensus.cc:3037] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:32.688045 24414 raft_consensus.cc:3037] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:32.690912 24414 raft_consensus.cc:2454] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3763b39b7a0a4417ac3e799d229eb150 in term 1.
I20260810 06:44:32.691156 23968 raft_consensus.cc:2454] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3763b39b7a0a4417ac3e799d229eb150 in term 1.
I20260810 06:44:32.691397 24088 leader_election.cc:302] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3763b39b7a0a4417ac3e799d229eb150, c59ad7fe4f7847a88023208774885744; no voters:
I20260810 06:44:32.691679 24543 raft_consensus.cc:2781] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:32.691836 24543 raft_consensus.cc:686] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [term 1 LEADER]: Becoming Leader. State: Replica: 3763b39b7a0a4417ac3e799d229eb150, State: Running, Role: LEADER
I20260810 06:44:32.692073 24543 consensus_queue.cc:237] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:32.693768 24582 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: TxnStatusTablet state changed. Reason: New leader 3763b39b7a0a4417ac3e799d229eb150. Latest consensus state: current_term: 1 leader_uuid: "3763b39b7a0a4417ac3e799d229eb150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.694250 24582 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:44:32.694975 24605 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:44:32.695226 24605 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:44:32.695163 23707 catalog_manager.cc:5495] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2). New cstate: current_term: 1 leader_uuid: "3763b39b7a0a4417ac3e799d229eb150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } health_report { overall_health: UNKNOWN } } }
I20260810 06:44:32.720284 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.714755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:35097 (local address 127.0.0.1:38854)
0810 06:44:32.715048 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.715063 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.715206 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.715647 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.715652 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.715664 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:44:32.715834 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.715842 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.717014 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.717020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.717985 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.717997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.718745 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.718752 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.718903 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.718963 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:44:32.719742 (+ 779us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:44:32.719764 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.719841 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.719986 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260810 06:44:32.720371 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.714732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.10.242.2:35097)
0810 06:44:32.714913 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.714920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.715140 (+ 220us) server_negotiation.cc:407] Connection header received
0810 06:44:32.715398 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.715403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.715489 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.715613 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:32.715962 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.716891 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.718116 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.718596 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.718712 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.719157 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:32.719635 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:32.719756 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.719875 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.719954 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:44:32.723937 24414 raft_consensus.cc:1261] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Refusing update from remote peer 3763b39b7a0a4417ac3e799d229eb150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:32.724380 23968 raft_consensus.cc:1261] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Refusing update from remote peer 3763b39b7a0a4417ac3e799d229eb150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:32.724897 24543 consensus_queue.cc:1035] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:32.725492 24543 consensus_queue.cc:1035] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:32.734100 24544 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: TxnStatusTablet state changed. Reason: New leader 3763b39b7a0a4417ac3e799d229eb150. Latest consensus state: current_term: 1 leader_uuid: "3763b39b7a0a4417ac3e799d229eb150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.734447 24544 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:44:32.739008 24582 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3763b39b7a0a4417ac3e799d229eb150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.739248 24582 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:44:32.739485 24544 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3763b39b7a0a4417ac3e799d229eb150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.739693 24544 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P 6a8adf2f1d144af59fa1868bd576e51d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:44:32.743336 24554 mvcc.cc:205] Tried to move back new op lower bound from 7316866141070733312 to 7316866140949049344. Current Snapshot: MvccSnapshot[applied={T|T < 7316866141070733312 or (T in {7316866141070733312})}]
I20260810 06:44:32.748346 24546 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: TxnStatusTablet state changed. Reason: New leader 3763b39b7a0a4417ac3e799d229eb150. Latest consensus state: current_term: 1 leader_uuid: "3763b39b7a0a4417ac3e799d229eb150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.748659 24546 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:44:32.750289 24582 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3763b39b7a0a4417ac3e799d229eb150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.750545 24582 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P 3763b39b7a0a4417ac3e799d229eb150: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:44:32.752305 24546 tablet_replica.cc:439] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3763b39b7a0a4417ac3e799d229eb150" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } }
I20260810 06:44:32.752537 24546 tablet_replica.cc:442] T 1ae11cee4f5f45019fd76e7ce9399dda P c59ad7fe4f7847a88023208774885744: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:44:32.753822 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.746348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.10.242.2:35097)
0810 06:44:32.746522 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.746528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.746547 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:44:32.746611 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.746616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.746695 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.747319 (+ 624us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:32.747470 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.748572 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.750619 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.751288 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.751910 (+ 622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.752009 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:32.752124 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:32.753417 (+ 1293us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.753470 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.753549 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:44:32.773910 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.766351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:50379 (local address 127.10.242.2:35097)
0810 06:44:32.766795 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.766801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.766821 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:44:32.766879 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.766884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.766967 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.767064 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:32.767698 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.768881 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.771414 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.772052 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.772186 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.772802 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:32.772983 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:32.773082 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.773270 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.773411 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":293}
I20260810 06:44:32.773943 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.765669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:35097 (local address 127.10.242.2:50379)
0810 06:44:32.766469 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:32.766494 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:32.766638 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:32.767097 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:32.767103 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:32.767119 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:44:32.767509 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.767521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.769034 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.769040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.771239 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:32.771255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.772262 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.772270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:32.772526 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.772606 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:44:32.773133 (+ 527us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:44:32.773142 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:44:32.773218 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 06:44:32.773440 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":50,"thread_start_us":743,"threads_started":1}
I20260810 06:44:32.785727 24413 raft_consensus.cc:1261] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Refusing update from remote peer 3763b39b7a0a4417ac3e799d229eb150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:32.785910 23968 raft_consensus.cc:1261] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Refusing update from remote peer 3763b39b7a0a4417ac3e799d229eb150: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:32.786633 24582 consensus_queue.cc:1035] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:32.787137 24582 consensus_queue.cc:1035] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:32.885577 23705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:32.885869 23705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:32.889170 23705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d: Sending DeleteTablet for 3 replicas of tablet 214c32bb1a1a424483d67d177f8ac498
I20260810 06:44:32.890028 23936 tablet_service.cc:1514] Processing DeleteTablet for tablet 214c32bb1a1a424483d67d177f8ac498 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:32 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:38638
I20260810 06:44:32.890184 24383 tablet_service.cc:1514] Processing DeleteTablet for tablet 214c32bb1a1a424483d67d177f8ac498 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:32 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:43182
I20260810 06:44:32.890377 24571 tablet_replica.cc:330] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: stopping tablet replica
I20260810 06:44:32.890395 24156 tablet_service.cc:1514] Processing DeleteTablet for tablet 214c32bb1a1a424483d67d177f8ac498 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:32 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:47814
I20260810 06:44:32.890585 24571 raft_consensus.cc:2229] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:32.890668 24569 tablet_replica.cc:330] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: stopping tablet replica
I20260810 06:44:32.890784 24571 raft_consensus.cc:2258] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:32.890889 24569 raft_consensus.cc:2229] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:44:32.890983 24570 tablet_replica.cc:330] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: stopping tablet replica
I20260810 06:44:32.891184 24570 raft_consensus.cc:2229] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:32.891374 24570 raft_consensus.cc:2258] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:32.891399 24569 raft_consensus.cc:2258] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:32.891983 24571 ts_tablet_manager.cc:1905] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:32.893306 24569 ts_tablet_manager.cc:1905] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:32.893512 24570 ts_tablet_manager.cc:1905] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:32.899060 24571 ts_tablet_manager.cc:1918] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:44:32.899232 24571 log.cc:1196] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/214c32bb1a1a424483d67d177f8ac498
I20260810 06:44:32.899631 24571 ts_tablet_manager.cc:1939] T 214c32bb1a1a424483d67d177f8ac498 P 6a8adf2f1d144af59fa1868bd576e51d: Deleting consensus metadata
I20260810 06:44:32.900673 23685 catalog_manager.cc:4841] TS 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675): tablet 214c32bb1a1a424483d67d177f8ac498 (table test-table [id=d138a3032c3347ac959907ab0bc58174]) successfully deleted
I20260810 06:44:32.901228 24570 ts_tablet_manager.cc:1918] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:44:32.901360 24570 log.cc:1196] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/214c32bb1a1a424483d67d177f8ac498
I20260810 06:44:32.901705 24570 ts_tablet_manager.cc:1939] T 214c32bb1a1a424483d67d177f8ac498 P c59ad7fe4f7847a88023208774885744: Deleting consensus metadata
I20260810 06:44:32.902576 23687 catalog_manager.cc:4841] TS c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777): tablet 214c32bb1a1a424483d67d177f8ac498 (table test-table [id=d138a3032c3347ac959907ab0bc58174]) successfully deleted
I20260810 06:44:32.909940 24569 ts_tablet_manager.cc:1918] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:44:32.910184 24569 log.cc:1196] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/214c32bb1a1a424483d67d177f8ac498
I20260810 06:44:32.910647 24569 ts_tablet_manager.cc:1939] T 214c32bb1a1a424483d67d177f8ac498 P 3763b39b7a0a4417ac3e799d229eb150: Deleting consensus metadata
I20260810 06:44:32.911666 23686 catalog_manager.cc:4841] TS 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097): tablet 214c32bb1a1a424483d67d177f8ac498 (table test-table [id=d138a3032c3347ac959907ab0bc58174]) successfully deleted
Aug 10 06:44:32 dist-test-slave-zj8q krb5kdc[23632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:44:32 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:32 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:44:32.969523 24599 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:32.957565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54384 (local address 127.10.242.62:40355)
0810 06:44:32.957762 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.957769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.958334 (+ 565us) server_negotiation.cc:407] Connection header received
0810 06:44:32.958528 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.958532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.958602 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.958709 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:32.960318 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.961025 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.962904 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.963381 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.963473 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.969262 (+ 5789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54384: BlockingRecv error: recv got EOF from 127.0.0.1:54384 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 10 06:44:32 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM
I20260810 06:44:32.986971 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:32.972511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.10.242.62:40355)
0810 06:44:32.972714 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:32.972721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:32.973105 (+ 384us) server_negotiation.cc:407] Connection header received
0810 06:44:32.973276 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:32.973280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:32.973354 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:32.973429 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:32.974845 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.975533 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.977262 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:32.977686 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:32.977773 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:32.982010 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:32.982015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:32.982019 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:32.982050 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:32.984445 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:32.985132 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:32.985136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:32.985138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:32.985278 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:32.985672 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:32.985681 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:32.985683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:32.985919 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:32.986114 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:32.986480 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 06:44:32.986640 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:44:32.990252 23705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:44:32.990983 23705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:44:32.998569 24383 tablet_service.cc:1467] Processing CreateTablet for tablet 6b9adfb1bd994f8bb272ceeef2334c31 (DEFAULT_TABLE table=test-table [id=19a07303564845199be3f39e4965a91b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:32.998608 23936 tablet_service.cc:1467] Processing CreateTablet for tablet 6b9adfb1bd994f8bb272ceeef2334c31 (DEFAULT_TABLE table=test-table [id=19a07303564845199be3f39e4965a91b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:32.998790 24156 tablet_service.cc:1467] Processing CreateTablet for tablet 6b9adfb1bd994f8bb272ceeef2334c31 (DEFAULT_TABLE table=test-table [id=19a07303564845199be3f39e4965a91b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:32.999024 23936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b9adfb1bd994f8bb272ceeef2334c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.999043 24383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b9adfb1bd994f8bb272ceeef2334c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:32.999269 24156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b9adfb1bd994f8bb272ceeef2334c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:33.005246 24532 tablet_bootstrap.cc:492] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: Bootstrap starting.
I20260810 06:44:33.005268 24530 tablet_bootstrap.cc:492] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: Bootstrap starting.
I20260810 06:44:33.006668 24530 tablet_bootstrap.cc:654] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:33.006914 24532 tablet_bootstrap.cc:654] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:33.009142 24530 tablet_bootstrap.cc:492] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: No bootstrap required, opened a new log
I20260810 06:44:33.009142 24532 tablet_bootstrap.cc:492] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: No bootstrap required, opened a new log
I20260810 06:44:33.009274 24530 ts_tablet_manager.cc:1397] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 06:44:33.009276 24532 ts_tablet_manager.cc:1397] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:44:33.009862 24530 raft_consensus.cc:348] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:33.010048 24530 raft_consensus.cc:374] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:33.009982 24532 raft_consensus.cc:348] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:33.010176 24530 raft_consensus.cc:729] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8adf2f1d144af59fa1868bd576e51d, State: Initialized, Role: FOLLOWER
I20260810 06:44:33.010416 24532 raft_consensus.cc:374] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:33.010392 24530 consensus_queue.cc:260] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:33.010577 24532 raft_consensus.cc:729] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad7fe4f7847a88023208774885744, State: Initialized, Role: FOLLOWER
I20260810 06:44:33.010841 24532 consensus_queue.cc:260] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:33.011610 24530 ts_tablet_manager.cc:1428] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:44:33.011703 24533 tablet_bootstrap.cc:492] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: Bootstrap starting.
I20260810 06:44:33.013280 24533 tablet_bootstrap.cc:654] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:33.015021 24532 ts_tablet_manager.cc:1428] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260810 06:44:33.018625 24533 tablet_bootstrap.cc:492] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: No bootstrap required, opened a new log
I20260810 06:44:33.018761 24533 ts_tablet_manager.cc:1397] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260810 06:44:33.019445 24533 raft_consensus.cc:348] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:33.019657 24533 raft_consensus.cc:374] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:33.019760 24533 raft_consensus.cc:729] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3763b39b7a0a4417ac3e799d229eb150, State: Initialized, Role: FOLLOWER
I20260810 06:44:33.019966 24533 consensus_queue.cc:260] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:33.020453 24533 ts_tablet_manager.cc:1428] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:44:33.040490 24544 raft_consensus.cc:482] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:33.040691 24544 raft_consensus.cc:504] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:33.041458 24544 leader_election.cc:288] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097), c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777)
I20260810 06:44:33.041911 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b9adfb1bd994f8bb272ceeef2334c31" candidate_uuid: "6a8adf2f1d144af59fa1868bd576e51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad7fe4f7847a88023208774885744" is_pre_election: true
I20260810 06:44:33.041903 24195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b9adfb1bd994f8bb272ceeef2334c31" candidate_uuid: "6a8adf2f1d144af59fa1868bd576e51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3763b39b7a0a4417ac3e799d229eb150" is_pre_election: true
I20260810 06:44:33.042145 24414 raft_consensus.cc:2454] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8adf2f1d144af59fa1868bd576e51d in term 0.
I20260810 06:44:33.042232 24195 raft_consensus.cc:2454] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a8adf2f1d144af59fa1868bd576e51d in term 0.
I20260810 06:44:33.042582 23860 leader_election.cc:302] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8adf2f1d144af59fa1868bd576e51d, c59ad7fe4f7847a88023208774885744; no voters:
I20260810 06:44:33.042856 24544 raft_consensus.cc:2781] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:44:33.042985 24544 raft_consensus.cc:482] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:33.043092 24544 raft_consensus.cc:3037] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:33.046029 24544 raft_consensus.cc:504] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:33.046674 24544 leader_election.cc:288] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [CANDIDATE]: Term 1 election: Requested vote from peers 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097), c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777)
I20260810 06:44:33.047067 24414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b9adfb1bd994f8bb272ceeef2334c31" candidate_uuid: "6a8adf2f1d144af59fa1868bd576e51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59ad7fe4f7847a88023208774885744"
I20260810 06:44:33.047118 24195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b9adfb1bd994f8bb272ceeef2334c31" candidate_uuid: "6a8adf2f1d144af59fa1868bd576e51d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3763b39b7a0a4417ac3e799d229eb150"
I20260810 06:44:33.047258 24414 raft_consensus.cc:3037] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:33.047348 24195 raft_consensus.cc:3037] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:33.049197 24414 raft_consensus.cc:2454] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8adf2f1d144af59fa1868bd576e51d in term 1.
I20260810 06:44:33.049525 23860 leader_election.cc:302] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a8adf2f1d144af59fa1868bd576e51d, c59ad7fe4f7847a88023208774885744; no voters:
I20260810 06:44:33.049784 24544 raft_consensus.cc:2781] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:33.050184 24544 raft_consensus.cc:686] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 1 LEADER]: Becoming Leader. State: Replica: 6a8adf2f1d144af59fa1868bd576e51d, State: Running, Role: LEADER
I20260810 06:44:33.050513 24544 consensus_queue.cc:237] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } }
I20260810 06:44:33.050714 24195 raft_consensus.cc:2454] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8adf2f1d144af59fa1868bd576e51d in term 1.
I20260810 06:44:33.053025 23707 catalog_manager.cc:5495] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1). New cstate: current_term: 1 leader_uuid: "6a8adf2f1d144af59fa1868bd576e51d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } health_report { overall_health: UNKNOWN } } }
I20260810 06:44:33.066612 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.061186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.10.242.1:42675)
0810 06:44:33.061686 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:33.061690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:33.061706 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:44:33.061755 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:33.061758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:33.061815 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:33.061891 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:33.062343 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.063070 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.064943 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.065327 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.065414 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.065911 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:33.066014 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:33.066072 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:33.066266 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 06:44:33.066328 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":259,"threads_started":1}
I20260810 06:44:33.069195 24414 raft_consensus.cc:1261] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Refusing update from remote peer 6a8adf2f1d144af59fa1868bd576e51d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:33.069561 24195 raft_consensus.cc:1261] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Refusing update from remote peer 6a8adf2f1d144af59fa1868bd576e51d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:33.069746 24544 consensus_queue.cc:1035] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:33.070250 24544 consensus_queue.cc:1035] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:33.085876 23705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:33.086193 23705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:33.089779 23705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d: Sending DeleteTablet for 3 replicas of tablet 6b9adfb1bd994f8bb272ceeef2334c31
I20260810 06:44:33.090626 24383 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b9adfb1bd994f8bb272ceeef2334c31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:33 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:43182
I20260810 06:44:33.090939 24156 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b9adfb1bd994f8bb272ceeef2334c31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:33 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:47814
I20260810 06:44:33.091071 24570 tablet_replica.cc:330] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: stopping tablet replica
I20260810 06:44:33.091099 23936 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b9adfb1bd994f8bb272ceeef2334c31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:33 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:38638
I20260810 06:44:33.091238 24569 tablet_replica.cc:330] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: stopping tablet replica
I20260810 06:44:33.091251 24570 raft_consensus.cc:2229] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:33.091403 24569 raft_consensus.cc:2229] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:33.091485 24570 raft_consensus.cc:2258] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:33.091619 24569 raft_consensus.cc:2258] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:33.091890 24383 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b9adfb1bd994f8bb272ceeef2334c31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:33 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:43182
I20260810 06:44:33.092166 24571 tablet_replica.cc:330] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: stopping tablet replica
I20260810 06:44:33.092350 24571 raft_consensus.cc:2229] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:44:33.092654 24571 raft_consensus.cc:2258] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:33.092912 24570 ts_tablet_manager.cc:1905] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:33.093736 24571 ts_tablet_manager.cc:1905] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:33.099784 24569 ts_tablet_manager.cc:1905] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:33.100035 24571 ts_tablet_manager.cc:1918] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:33.100153 24571 log.cc:1196] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/6b9adfb1bd994f8bb272ceeef2334c31
I20260810 06:44:33.099946 23707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 06:44:33.100548 24571 ts_tablet_manager.cc:1939] T 6b9adfb1bd994f8bb272ceeef2334c31 P 6a8adf2f1d144af59fa1868bd576e51d: Deleting consensus metadata
W20260810 06:44:33.100618 23707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:44:33.101536 23685 catalog_manager.cc:4841] TS 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675): tablet 6b9adfb1bd994f8bb272ceeef2334c31 (table test-table [id=19a07303564845199be3f39e4965a91b]) successfully deleted
I20260810 06:44:33.104921 24569 ts_tablet_manager.cc:1918] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:33.105019 24569 log.cc:1196] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/6b9adfb1bd994f8bb272ceeef2334c31
I20260810 06:44:33.105350 24569 ts_tablet_manager.cc:1939] T 6b9adfb1bd994f8bb272ceeef2334c31 P 3763b39b7a0a4417ac3e799d229eb150: Deleting consensus metadata
I20260810 06:44:33.106413 23686 catalog_manager.cc:4841] TS 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097): tablet 6b9adfb1bd994f8bb272ceeef2334c31 (table test-table [id=19a07303564845199be3f39e4965a91b]) successfully deleted
I20260810 06:44:33.107429 24570 ts_tablet_manager.cc:1918] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:44:33.107645 24570 log.cc:1196] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/6b9adfb1bd994f8bb272ceeef2334c31
I20260810 06:44:33.108121 24570 ts_tablet_manager.cc:1939] T 6b9adfb1bd994f8bb272ceeef2334c31 P c59ad7fe4f7847a88023208774885744: Deleting consensus metadata
I20260810 06:44:33.109575 23687 catalog_manager.cc:4841] TS c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777): tablet 6b9adfb1bd994f8bb272ceeef2334c31 (table test-table [id=19a07303564845199be3f39e4965a91b]) successfully deleted
W20260810 06:44:33.110008 23687 catalog_manager.cc:4808] TS c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777): delete failed for tablet 6b9adfb1bd994f8bb272ceeef2334c31 because the tablet was not found. No further retry: Not found: Tablet not found: 6b9adfb1bd994f8bb272ceeef2334c31
I20260810 06:44:33.112532 24383 tablet_service.cc:1467] Processing CreateTablet for tablet 4d229008496144cca543ae85e4cfcfcd (DEFAULT_TABLE table=test-table [id=8de1fc17ec954522a87bd46bb104f50d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:33.112967 24383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d229008496144cca543ae85e4cfcfcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:33.113113 23936 tablet_service.cc:1467] Processing CreateTablet for tablet 4d229008496144cca543ae85e4cfcfcd (DEFAULT_TABLE table=test-table [id=8de1fc17ec954522a87bd46bb104f50d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:33.113570 23936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d229008496144cca543ae85e4cfcfcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:33.114573 24156 tablet_service.cc:1467] Processing CreateTablet for tablet 4d229008496144cca543ae85e4cfcfcd (DEFAULT_TABLE table=test-table [id=8de1fc17ec954522a87bd46bb104f50d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:44:33.114974 24156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d229008496144cca543ae85e4cfcfcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:33.118705 24532 tablet_bootstrap.cc:492] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: Bootstrap starting.
I20260810 06:44:33.120237 24532 tablet_bootstrap.cc:654] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:33.121338 24530 tablet_bootstrap.cc:492] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: Bootstrap starting.
I20260810 06:44:33.121843 24533 tablet_bootstrap.cc:492] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: Bootstrap starting.
I20260810 06:44:33.122956 24530 tablet_bootstrap.cc:654] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:33.123562 24533 tablet_bootstrap.cc:654] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:33.125125 24530 tablet_bootstrap.cc:492] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: No bootstrap required, opened a new log
I20260810 06:44:33.125295 24530 ts_tablet_manager.cc:1397] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:44:33.125803 24533 tablet_bootstrap.cc:492] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: No bootstrap required, opened a new log
I20260810 06:44:33.125976 24533 ts_tablet_manager.cc:1397] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:44:33.126098 24530 raft_consensus.cc:348] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } }
I20260810 06:44:33.126502 24530 raft_consensus.cc:374] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:33.126792 24530 raft_consensus.cc:729] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8adf2f1d144af59fa1868bd576e51d, State: Initialized, Role: FOLLOWER
I20260810 06:44:33.126917 24533 raft_consensus.cc:348] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } }
I20260810 06:44:33.127169 24533 raft_consensus.cc:374] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:33.127112 24530 consensus_queue.cc:260] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } }
I20260810 06:44:33.127275 24533 raft_consensus.cc:729] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3763b39b7a0a4417ac3e799d229eb150, State: Initialized, Role: FOLLOWER
I20260810 06:44:33.127599 24533 consensus_queue.cc:260] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } }
I20260810 06:44:33.127765 24530 ts_tablet_manager.cc:1428] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:44:33.128211 24533 ts_tablet_manager.cc:1428] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:44:33.129482 24532 tablet_bootstrap.cc:492] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: No bootstrap required, opened a new log
I20260810 06:44:33.129657 24532 ts_tablet_manager.cc:1397] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260810 06:44:33.131266 24532 raft_consensus.cc:348] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } }
I20260810 06:44:33.131542 24532 raft_consensus.cc:374] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:33.131631 24532 raft_consensus.cc:729] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59ad7fe4f7847a88023208774885744, State: Initialized, Role: FOLLOWER
I20260810 06:44:33.131918 24532 consensus_queue.cc:260] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } }
I20260810 06:44:33.133033 24532 ts_tablet_manager.cc:1428] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:44:33.155198 24546 raft_consensus.cc:482] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:44:33.155398 24546 raft_consensus.cc:504] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } }
I20260810 06:44:33.157132 24546 leader_election.cc:288] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675), 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097)
I20260810 06:44:33.164774 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.157724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:49071 (local address 127.10.242.1:42675)
0810 06:44:33.157893 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:33.157900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:33.157978 (+ 78us) server_negotiation.cc:407] Connection header received
0810 06:44:33.158301 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:33.158307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:33.158383 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:33.158446 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:33.159149 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.159965 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.162359 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.163609 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.163803 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.163876 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:33.164327 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 06:44:33.164476 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:44:33.164813 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.157548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:54703 (local address 127.10.242.2:35097)
0810 06:44:33.157736 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:33.157743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:33.158652 (+ 909us) server_negotiation.cc:407] Connection header received
0810 06:44:33.158911 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:33.158917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:33.159039 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:33.159111 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:44:33.159879 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.160645 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.162408 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.163619 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.163815 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.163877 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:33.164431 (+ 554us) server_negotiation.cc:299] Negotiation successful
0810 06:44:33.164567 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:44:33.164857 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.157339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:35097 (local address 127.10.242.3:54703)
0810 06:44:33.158552 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:33.158570 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:33.158762 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:33.159307 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:33.159312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:33.159338 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:33.159698 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:33.159709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.160780 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.160784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:33.162215 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:33.162228 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.164026 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.164032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:33.164211 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.164243 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:44:33.164382 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 06:44:33.164504 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1095,"mutex_wait_us":4,"thread_start_us":1049,"threads_started":1}
I20260810 06:44:33.165542 24672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.157291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:42675 (local address 127.10.242.3:49071)
0810 06:44:33.157862 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:33.157880 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:33.158091 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:33.158644 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:33.158650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:33.158693 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:44:33.158967 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:33.158980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.160099 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.160104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:33.162196 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:33.162212 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.163801 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.163807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:33.164025 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.164114 (+ 89us) client_negotiation.cc:769] Sending connection context
0810 06:44:33.165085 (+ 971us) client_negotiation.cc:240] Negotiation successful
0810 06:44:33.165210 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":66,"thread_start_us":457,"threads_started":1}
I20260810 06:44:33.165587 24195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d229008496144cca543ae85e4cfcfcd" candidate_uuid: "c59ad7fe4f7847a88023208774885744" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3763b39b7a0a4417ac3e799d229eb150" is_pre_election: true
I20260810 06:44:33.165845 24195 raft_consensus.cc:2454] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59ad7fe4f7847a88023208774885744 in term 0.
I20260810 06:44:33.166232 23968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d229008496144cca543ae85e4cfcfcd" candidate_uuid: "c59ad7fe4f7847a88023208774885744" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8adf2f1d144af59fa1868bd576e51d" is_pre_election: true
I20260810 06:44:33.166460 23968 raft_consensus.cc:2454] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c59ad7fe4f7847a88023208774885744 in term 0.
I20260810 06:44:33.166405 24299 leader_election.cc:302] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3763b39b7a0a4417ac3e799d229eb150, c59ad7fe4f7847a88023208774885744; no voters:
I20260810 06:44:33.166821 24546 raft_consensus.cc:2781] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:44:33.166944 24546 raft_consensus.cc:482] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:44:33.167038 24546 raft_consensus.cc:3037] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:33.170235 24546 raft_consensus.cc:504] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } }
I20260810 06:44:33.170842 24546 leader_election.cc:288] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [CANDIDATE]: Term 1 election: Requested vote from peers 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675), 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097)
I20260810 06:44:33.171352 23968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d229008496144cca543ae85e4cfcfcd" candidate_uuid: "c59ad7fe4f7847a88023208774885744" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a8adf2f1d144af59fa1868bd576e51d"
I20260810 06:44:33.171545 23968 raft_consensus.cc:3037] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:33.171520 24195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d229008496144cca543ae85e4cfcfcd" candidate_uuid: "c59ad7fe4f7847a88023208774885744" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3763b39b7a0a4417ac3e799d229eb150"
I20260810 06:44:33.171754 24195 raft_consensus.cc:3037] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:33.174192 24195 raft_consensus.cc:2454] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59ad7fe4f7847a88023208774885744 in term 1.
I20260810 06:44:33.174355 23968 raft_consensus.cc:2454] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c59ad7fe4f7847a88023208774885744 in term 1.
I20260810 06:44:33.174652 24299 leader_election.cc:302] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3763b39b7a0a4417ac3e799d229eb150, c59ad7fe4f7847a88023208774885744; no voters:
I20260810 06:44:33.174917 24546 raft_consensus.cc:2781] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:33.175369 24546 raft_consensus.cc:686] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 1 LEADER]: Becoming Leader. State: Replica: c59ad7fe4f7847a88023208774885744, State: Running, Role: LEADER
I20260810 06:44:33.175774 24546 consensus_queue.cc:237] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } }
I20260810 06:44:33.179282 23705 catalog_manager.cc:5495] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 reported cstate change: term changed from 0 to 1, leader changed from <none> to c59ad7fe4f7847a88023208774885744 (127.10.242.3). New cstate: current_term: 1 leader_uuid: "c59ad7fe4f7847a88023208774885744" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c59ad7fe4f7847a88023208774885744" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 40777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 } health_report { overall_health: UNKNOWN } } }
I20260810 06:44:33.209532 24676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.203904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.10.242.3:40777)
0810 06:44:33.204489 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:33.204493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:33.204510 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:44:33.204584 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:33.204587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:33.204656 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:33.204798 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:33.205235 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.206010 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.207332 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.207762 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.207869 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.208322 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:33.208918 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:33.209054 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:33.209137 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 06:44:33.209221 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":357,"threads_started":1}
I20260810 06:44:33.224670 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.219278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:44837 (local address 127.10.242.2:35097)
0810 06:44:33.219480 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:33.219486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:33.219989 (+ 503us) server_negotiation.cc:407] Connection header received
0810 06:44:33.220169 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:33.220175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:33.220260 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:33.220358 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:33.220760 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.221626 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.222967 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.223420 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.223542 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.223934 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:33.224051 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:33.224121 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:33.224262 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 06:44:33.224341 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:44:33.224709 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.219267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:35097 (local address 127.10.242.3:44837)
0810 06:44:33.219903 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:33.219918 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:33.220051 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:33.220366 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:33.220369 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:33.220378 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:44:33.220621 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:33.220628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.221748 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.221754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:33.222837 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:33.222849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.223600 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.223607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:33.223721 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.223769 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:44:33.224144 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:44:33.224154 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 06:44:33.224224 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:44:33.224350 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":326,"threads_started":1}
I20260810 06:44:33.233403 24195 raft_consensus.cc:1261] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Refusing update from remote peer c59ad7fe4f7847a88023208774885744: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:33.233703 23968 raft_consensus.cc:1261] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Refusing update from remote peer c59ad7fe4f7847a88023208774885744: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:44:33.234263 24674 consensus_queue.cc:1035] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3763b39b7a0a4417ac3e799d229eb150" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:33.234988 24674 consensus_queue.cc:1035] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a8adf2f1d144af59fa1868bd576e51d" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 42675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:44:33.277449 24676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.272029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:39865 (local address 127.10.242.3:40777)
0810 06:44:33.272188 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:33.272193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:33.272211 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:44:33.272266 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:33.272270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:33.272348 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:33.272440 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:44:33.272827 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.273942 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.275962 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.276352 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.276441 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.276870 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:44:33.276960 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:44:33.277015 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:33.277164 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 06:44:33.277232 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 06:44:33.277616 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.271817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:40777 (local address 127.10.242.2:39865)
0810 06:44:33.271989 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:33.272010 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:33.272132 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:33.272472 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:33.272477 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:33.272490 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:44:33.272698 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:33.272711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.274089 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.274094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:33.275819 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:33.275836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.276497 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.276504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:33.276685 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.276737 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:44:33.277052 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:44:33.277067 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 06:44:33.277153 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:44:33.277299 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:44:33.315996 23707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:33.316260 23707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:44:33.319802 23707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e9087eee7a8481aa1aa73c445b96f0d: Sending DeleteTablet for 3 replicas of tablet 4d229008496144cca543ae85e4cfcfcd
I20260810 06:44:33.320538 24383 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d229008496144cca543ae85e4cfcfcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:33 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:43182
I20260810 06:44:33.320811 24570 tablet_replica.cc:330] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: stopping tablet replica
I20260810 06:44:33.321039 24570 raft_consensus.cc:2229] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:44:33.321552 24570 raft_consensus.cc:2258] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:33.322265 23936 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d229008496144cca543ae85e4cfcfcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:33 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:38638
I20260810 06:44:33.322556 24571 tablet_replica.cc:330] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: stopping tablet replica
I20260810 06:44:33.322754 24571 raft_consensus.cc:2229] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:33.322932 24571 raft_consensus.cc:2258] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:33.323078 24570 ts_tablet_manager.cc:1905] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:33.326423 24156 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d229008496144cca543ae85e4cfcfcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:44:33 UTC) from {username='oryx', principal='oryx/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:47814
I20260810 06:44:33.326751 24569 tablet_replica.cc:330] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: stopping tablet replica
I20260810 06:44:33.326961 24569 raft_consensus.cc:2229] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:44:33.327157 24569 raft_consensus.cc:2258] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:44:33.328616 24569 ts_tablet_manager.cc:1905] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:33.328802 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 23824
I20260810 06:44:33.332880 24571 ts_tablet_manager.cc:1905] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:44:33.337648 24570 ts_tablet_manager.cc:1918] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:44:33.337896 24570 log.cc:1196] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/4d229008496144cca543ae85e4cfcfcd
I20260810 06:44:33.338501 24570 ts_tablet_manager.cc:1939] T 4d229008496144cca543ae85e4cfcfcd P c59ad7fe4f7847a88023208774885744: Deleting consensus metadata
I20260810 06:44:33.339720 23687 catalog_manager.cc:4841] TS c59ad7fe4f7847a88023208774885744 (127.10.242.3:40777): tablet 4d229008496144cca543ae85e4cfcfcd (table test-table [id=8de1fc17ec954522a87bd46bb104f50d]) successfully deleted
I20260810 06:44:33.339962 24571 ts_tablet_manager.cc:1918] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:44:33.340111 24571 log.cc:1196] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/4d229008496144cca543ae85e4cfcfcd
I20260810 06:44:33.340497 24571 ts_tablet_manager.cc:1939] T 4d229008496144cca543ae85e4cfcfcd P 6a8adf2f1d144af59fa1868bd576e51d: Deleting consensus metadata
I20260810 06:44:33.341584 23685 catalog_manager.cc:4841] TS 6a8adf2f1d144af59fa1868bd576e51d (127.10.242.1:42675): tablet 4d229008496144cca543ae85e4cfcfcd (table test-table [id=8de1fc17ec954522a87bd46bb104f50d]) successfully deleted
I20260810 06:44:33.343416 24569 ts_tablet_manager.cc:1918] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:44:33.343638 24569 log.cc:1196] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/4d229008496144cca543ae85e4cfcfcd
I20260810 06:44:33.344122 24569 ts_tablet_manager.cc:1939] T 4d229008496144cca543ae85e4cfcfcd P 3763b39b7a0a4417ac3e799d229eb150: Deleting consensus metadata
I20260810 06:44:33.345147 23686 catalog_manager.cc:4841] TS 3763b39b7a0a4417ac3e799d229eb150 (127.10.242.2:35097): tablet 4d229008496144cca543ae85e4cfcfcd (table test-table [id=8de1fc17ec954522a87bd46bb104f50d]) successfully deleted
Aug 10 06:44:33 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.1@KRBTEST.COM
I20260810 06:44:33.355456 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.330418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.10.242.1:42675)
0810 06:44:33.330584 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:33.330590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:33.340619 (+ 10029us) server_negotiation.cc:407] Connection header received
0810 06:44:33.340903 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:33.340910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:33.340988 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:33.341063 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:33.342788 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.343823 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.345571 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.345972 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.346083 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.349905 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:33.349918 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:33.349933 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:33.349962 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:33.352618 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:33.353245 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:33.353252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:33.353254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:33.353372 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:33.353725 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:33.353729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:33.353731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:33.354113 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:33.354448 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:33.354918 (+ 470us) server_negotiation.cc:299] Negotiation successful
0810 06:44:33.355139 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:44:33.356029 24048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:33.506551 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 23824
I20260810 06:44:33.536279 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 24057
Aug 10 06:44:33 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.2@KRBTEST.COM
I20260810 06:44:33.551306 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.537148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38884 (local address 127.10.242.2:35097)
0810 06:44:33.537345 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:33.537350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:33.537369 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:44:33.537569 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:33.537575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:33.537644 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:33.537742 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:33.539016 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.539754 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.541001 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.541544 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.541657 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.545572 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:33.545585 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:33.545601 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:33.545643 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:33.548569 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:33.549204 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:33.549211 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:33.549214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:33.549362 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:33.549729 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:33.549733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:33.549735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:33.550092 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:33.550467 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:33.550703 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 06:44:33.550940 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:44:33.552171 24262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:33.700227 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 24057
I20260810 06:44:33.731352 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 24273
Aug 10 06:44:33 dist-test-slave-zj8q krb5kdc[23632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.242.3@KRBTEST.COM
I20260810 06:44:33.746407 24676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:33.732103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.10.242.3:40777)
0810 06:44:33.732319 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:33.732325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:33.732390 (+ 65us) server_negotiation.cc:407] Connection header received
0810 06:44:33.732581 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:33.732587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:33.732669 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:33.732774 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:33.734014 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.734812 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.736011 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:33.736456 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:33.736571 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:33.740713 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:33.740728 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:33.740746 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:33.740780 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:33.743784 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:33.744402 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:33.744411 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:33.744413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:33.744546 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:33.744901 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:33.744905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:33.744908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:33.745275 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:33.745616 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:33.745869 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 06:44:33.746086 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:44:33.747156 24485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:33.884124 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 24273
I20260810 06:44:33.914848 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 23653
I20260810 06:44:33.915983 23769 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:34.035056 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 23653
2026-08-10T06:44:34Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6695 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:44:34 dist-test-slave-zj8q krb5kdc[24701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:44:34 dist-test-slave-zj8q krb5kdc[24701](info): set up 2 sockets
Aug 10 06:44:34 dist-test-slave-zj8q krb5kdc[24701](info): commencing operation
krb5kdc: starting...
W20260810 06:44:35.270418 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:44:35 dist-test-slave-zj8q krb5kdc[24701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:44:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:44:35.505118 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:41349
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:40119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:41349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:35.827343 24720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:35.827654 24720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:35.827792 24720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:35.834755 24720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:35.834865 24720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:35.834944 24720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:35.835021 24720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:35.835088 24720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:35.844545 24720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:41349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:41349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:35.845789 24720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:35.847080 24720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:35.858248 24728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:35.858188 24734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:35.858210 24729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:35.859068 24720 server_base.cc:1034] running on GCE node
I20260810 06:44:35.859769 24720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:35.861025 24720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:35.862250 24720 hybrid_clock.cc:648] HybridClock initialized: now 1786344275862192 us; error 38 us; skew 500 ppm
I20260810 06:44:35.862581 24720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T06:44:35Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1891 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:44:36 dist-test-slave-zj8q krb5kdc[24747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:44:36 dist-test-slave-zj8q krb5kdc[24747](info): set up 2 sockets
Aug 10 06:44:36 dist-test-slave-zj8q krb5kdc[24747](info): commencing operation
krb5kdc: starting...
W20260810 06:44:37.151608 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:44:37 dist-test-slave-zj8q krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:44:37.387121 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34183
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:34183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:37.706981 24768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:37.707257 24768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:37.707382 24768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:37.714313 24768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:37.714403 24768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:37.714478 24768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:37.714556 24768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:37.714617 24768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:37.723866 24768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:34183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:37.725090 24768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:37.726359 24768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:37.737637 24778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:37.737620 24782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:37.737650 24779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:37.738385 24768 server_base.cc:1034] running on GCE node
I20260810 06:44:37.739144 24768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:37.740437 24768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:37.741640 24768 hybrid_clock.cc:648] HybridClock initialized: now 1786344277741593 us; error 28 us; skew 500 ppm
I20260810 06:44:37.741966 24768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:37 dist-test-slave-zj8q krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344277, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:37.748862 24768 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:44:37.750392 24768 webserver.cc:460] Webserver started at http://127.10.242.62:46577/ using document root <none> and password file <none>
I20260810 06:44:37.751046 24768 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:37.751163 24768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:37.751457 24768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:37.754371 24768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "623b0c8909f64a6d8451e670b171063e"
format_stamp: "Formatted at 2026-08-10 06:44:37 on dist-test-slave-zj8q"
server_key: "1144231fe420e1588cfba541faa95378"
server_key_iv: "f95752c0c25a07996005435742ca034d"
server_key_version: "encryptionkey@0"
I20260810 06:44:37.755110 24768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "623b0c8909f64a6d8451e670b171063e"
format_stamp: "Formatted at 2026-08-10 06:44:37 on dist-test-slave-zj8q"
server_key: "1144231fe420e1588cfba541faa95378"
server_key_iv: "f95752c0c25a07996005435742ca034d"
server_key_version: "encryptionkey@0"
I20260810 06:44:37.760076 24768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:44:37.763285 24795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:37.764489 24768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:44:37.764678 24768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "623b0c8909f64a6d8451e670b171063e"
format_stamp: "Formatted at 2026-08-10 06:44:37 on dist-test-slave-zj8q"
server_key: "1144231fe420e1588cfba541faa95378"
server_key_iv: "f95752c0c25a07996005435742ca034d"
server_key_version: "encryptionkey@0"
I20260810 06:44:37.764850 24768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:37.777771 24768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:37.781687 24768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:37.781935 24768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:37.798913 24768 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:34183
I20260810 06:44:37.798935 24885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:34183 every 8 connection(s)
I20260810 06:44:37.800179 24768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:37.803730 24886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:37.807219 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 24768
I20260810 06:44:37.807405 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:37.808034 11208 external_mini_cluster.cc:1428] Setting key 3b6e0935ce0acb72a6d18f6bd0837952
I20260810 06:44:37.814374 24886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: Bootstrap starting.
I20260810 06:44:37.817816 24886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:37.818975 24886 log.cc:824] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:37.822077 24886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: No bootstrap required, opened a new log
Aug 10 06:44:37 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:37.830946 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:37.809833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.10.242.62:34183)
0810 06:44:37.810603 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:37.810614 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:37.810658 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:44:37.812894 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:37.812916 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:37.813253 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:37.813537 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:37.815474 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:37.816760 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:37.818475 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:37.819153 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:37.819286 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:37.823920 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:37.823928 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:37.823945 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:37.823987 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:37.827013 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:37.827820 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:37.827827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:37.827832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:37.827993 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:37.828378 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:37.828381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:37.828383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:37.828906 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:37.829131 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:37.829574 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 06:44:37.829927 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":355,"threads_started":1}
I20260810 06:44:37.830796 24886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } }
I20260810 06:44:37.831187 24886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:37.831297 24886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 623b0c8909f64a6d8451e670b171063e, State: Initialized, Role: FOLLOWER
I20260810 06:44:37.831866 24886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } }
I20260810 06:44:37.832094 24886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:37.832264 24886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:37.832435 24886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:37.834703 24886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } }
I20260810 06:44:37.835165 24886 leader_election.cc:302] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 623b0c8909f64a6d8451e670b171063e; no voters:
I20260810 06:44:37.835762 24886 leader_election.cc:288] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:37.835870 24892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:37.836495 24892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 1 LEADER]: Becoming Leader. State: Replica: 623b0c8909f64a6d8451e670b171063e, State: Running, Role: LEADER
I20260810 06:44:37.837018 24892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } }
I20260810 06:44:37.837882 24886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:37.839809 24895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 623b0c8909f64a6d8451e670b171063e. Latest consensus state: current_term: 1 leader_uuid: "623b0c8909f64a6d8451e670b171063e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } } }
I20260810 06:44:37.840200 24895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:37.840189 24893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "623b0c8909f64a6d8451e670b171063e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } } }
I20260810 06:44:37.840533 24893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:37.846743 24906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:37.852308 24906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:37.860466 24906 catalog_manager.cc:1347] Generated new cluster ID: 3bb22910a7524026bcdd0772a86e441e
I20260810 06:44:37.860569 24906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:37.899326 24906 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:37.901139 24906 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:37.912154 24906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: Generated new TSK 0
I20260810 06:44:37.913138 24906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:44:37.973975 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34183
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:38.301885 24929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:38.302170 24929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:38.302315 24929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:38.309381 24929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:38.309486 24929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:38.309648 24929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:44:38.319118 24929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:38.320580 24929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:38.322279 24929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:38.335170 24938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:38.335222 24936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:38.335247 24941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:38.335867 24929 server_base.cc:1034] running on GCE node
I20260810 06:44:38.336462 24929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:38.337262 24929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:38.338459 24929 hybrid_clock.cc:648] HybridClock initialized: now 1786344278338414 us; error 27 us; skew 500 ppm
I20260810 06:44:38.338773 24929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:38 dist-test-slave-zj8q krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344278, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:38.341789 24929 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:44:38.343178 24929 webserver.cc:460] Webserver started at http://127.10.242.1:34855/ using document root <none> and password file <none>
I20260810 06:44:38.343786 24929 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:38.343884 24929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:38.344146 24929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:38.347078 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "d0aeed5737e94cd7abd6c97197ce397a"
format_stamp: "Formatted at 2026-08-10 06:44:38 on dist-test-slave-zj8q"
server_key: "372db909affc3b307336a82245276ba9"
server_key_iv: "e211889cf397d189b415f21fc70da394"
server_key_version: "encryptionkey@0"
I20260810 06:44:38.347842 24929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "d0aeed5737e94cd7abd6c97197ce397a"
format_stamp: "Formatted at 2026-08-10 06:44:38 on dist-test-slave-zj8q"
server_key: "372db909affc3b307336a82245276ba9"
server_key_iv: "e211889cf397d189b415f21fc70da394"
server_key_version: "encryptionkey@0"
I20260810 06:44:38.352695 24929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:44:38.355937 24954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:38.357154 24929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:44:38.357303 24929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "d0aeed5737e94cd7abd6c97197ce397a"
format_stamp: "Formatted at 2026-08-10 06:44:38 on dist-test-slave-zj8q"
server_key: "372db909affc3b307336a82245276ba9"
server_key_iv: "e211889cf397d189b415f21fc70da394"
server_key_version: "encryptionkey@0"
I20260810 06:44:38.357472 24929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:38.377403 24929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:38.381119 24929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:38.381367 24929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:38.382272 24929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:38.383509 24929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:38.383592 24929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:38.383716 24929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:38.383771 24929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:38.413285 24929 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:42353
I20260810 06:44:38.413349 25142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:42353 every 8 connection(s)
I20260810 06:44:38.414559 24929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:38.416584 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 24929
I20260810 06:44:38.416766 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:44:38.417541 11208 external_mini_cluster.cc:1428] Setting key 1d07932385d6111a591c82086f0d4183
Aug 10 06:44:38 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344278, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:38.439759 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:38.417027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:49379 (local address 127.10.242.62:34183)
0810 06:44:38.417511 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:38.417518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:38.418566 (+ 1048us) server_negotiation.cc:407] Connection header received
0810 06:44:38.420396 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:38.420402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:38.420478 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:38.420587 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:38.421990 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:38.422963 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:38.424541 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:38.425082 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:38.425209 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:38.429500 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:38.429533 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:38.429539 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:38.429584 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:38.432894 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:38.433898 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:38.433905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:38.433908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:38.434074 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:38.434629 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:38.434634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:38.434636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:38.434982 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:38.435106 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:38.439213 (+ 4107us) server_negotiation.cc:299] Negotiation successful
0810 06:44:38.439488 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":226,"threads_started":1}
I20260810 06:44:38.441143 25150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:38.417592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:49379)
0810 06:44:38.418412 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:38.418457 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:38.420140 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:38.420768 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:38.420775 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:38.421278 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:38.421821 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:38.421838 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:38.423132 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:38.423136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:38.424395 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:38.424409 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:38.425276 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:38.425281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:38.425466 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:38.426077 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:38.426103 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:38.429215 (+ 3112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:38.433081 (+ 3866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:38.433092 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:38.433108 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:38.433739 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:38.434235 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:38.434240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:38.434244 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:38.434512 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:38.438290 (+ 3778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:38.438299 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:38.438769 (+ 470us) client_negotiation.cc:769] Sending connection context
0810 06:44:38.439088 (+ 319us) client_negotiation.cc:240] Negotiation successful
0810 06:44:38.439728 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":291,"threads_started":1}
I20260810 06:44:38.443176 25144 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34183
I20260810 06:44:38.443526 25144 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:38.444448 25144 heartbeater.cc:511] Master 127.10.242.62:34183 requested a full tablet report, sending...
I20260810 06:44:38.447058 24826 ts_manager.cc:194] Registered new tserver with Master: d0aeed5737e94cd7abd6c97197ce397a (127.10.242.1:42353)
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
I20260810 06:44:38.448945 24826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:49379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:44:38.483496 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34183
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:38.804270 25156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:38.804563 25156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:38.804709 25156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:38.811638 25156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:38.811733 25156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:38.811868 25156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:38.820986 25156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:38.822181 25156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:38.823426 25156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:38.835925 25166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:38.836131 25174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:38.836367 25164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:38.836619 25156 server_base.cc:1034] running on GCE node
I20260810 06:44:38.837169 25156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:38.837980 25156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:38.839171 25156 hybrid_clock.cc:648] HybridClock initialized: now 1786344278839123 us; error 54 us; skew 500 ppm
I20260810 06:44:38.839491 25156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:38 dist-test-slave-zj8q krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344278, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:38.842437 25156 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:44:38.843734 25156 webserver.cc:460] Webserver started at http://127.10.242.2:38777/ using document root <none> and password file <none>
I20260810 06:44:38.844370 25156 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:38.844487 25156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:38.844782 25156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:38.847641 25156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "b7c3f8a765fb487393c28d9f9db02e93"
format_stamp: "Formatted at 2026-08-10 06:44:38 on dist-test-slave-zj8q"
server_key: "0d709e0862335f6e6559c39481028fd0"
server_key_iv: "03c332250c2d3a62cebf55f8fa806a81"
server_key_version: "encryptionkey@0"
I20260810 06:44:38.848382 25156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "b7c3f8a765fb487393c28d9f9db02e93"
format_stamp: "Formatted at 2026-08-10 06:44:38 on dist-test-slave-zj8q"
server_key: "0d709e0862335f6e6559c39481028fd0"
server_key_iv: "03c332250c2d3a62cebf55f8fa806a81"
server_key_version: "encryptionkey@0"
I20260810 06:44:38.853220 25156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 06:44:38.856431 25186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:38.857616 25156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:44:38.857779 25156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "b7c3f8a765fb487393c28d9f9db02e93"
format_stamp: "Formatted at 2026-08-10 06:44:38 on dist-test-slave-zj8q"
server_key: "0d709e0862335f6e6559c39481028fd0"
server_key_iv: "03c332250c2d3a62cebf55f8fa806a81"
server_key_version: "encryptionkey@0"
I20260810 06:44:38.857956 25156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:38.872074 25156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:38.875898 25156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:38.876140 25156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:38.877013 25156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:38.878289 25156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:38.878367 25156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:38.878480 25156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:38.878536 25156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:38.908260 25156 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:34913
I20260810 06:44:38.908311 25379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:34913 every 8 connection(s)
I20260810 06:44:38.909480 25156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:38.916326 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 25156
I20260810 06:44:38.916487 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:44:38.916920 11208 external_mini_cluster.cc:1428] Setting key 275ab422481975444f73e9beab28a5fa
Aug 10 06:44:38 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344278, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:38.928323 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:38.911898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:43403 (local address 127.10.242.62:34183)
0810 06:44:38.912078 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:38.912082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:38.913241 (+ 1159us) server_negotiation.cc:407] Connection header received
0810 06:44:38.915059 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:38.915065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:38.915146 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:38.915235 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:38.916751 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:38.917486 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:38.919311 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:38.919735 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:38.919844 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:38.923071 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:38.923075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:38.923079 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:38.923112 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:38.925402 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:38.926088 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:38.926092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:38.926094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:38.926260 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:38.926668 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:38.926674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:38.926684 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:38.926947 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:38.927078 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:38.927761 (+ 683us) server_negotiation.cc:299] Negotiation successful
0810 06:44:38.927988 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:44:38.929214 25385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:38.912386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:43403)
0810 06:44:38.913078 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:38.913115 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:38.914791 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:38.915472 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:38.915480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:38.915996 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:38.916580 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:38.916604 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:38.917634 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:38.917639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:38.919163 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:38.919187 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:38.919953 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:38.919961 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:38.920158 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:38.920798 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:38.920820 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:38.922861 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:38.925554 (+ 2693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:38.925564 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:38.925576 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:38.925963 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:38.926362 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:38.926368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:38.926372 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:38.926552 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:38.927070 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:38.927078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:38.927393 (+ 315us) client_negotiation.cc:769] Sending connection context
0810 06:44:38.927619 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 06:44:38.928107 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":263,"threads_started":1}
I20260810 06:44:38.930629 25381 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34183
I20260810 06:44:38.930954 25381 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:38.931653 25381 heartbeater.cc:511] Master 127.10.242.62:34183 requested a full tablet report, sending...
I20260810 06:44:38.933048 24826 ts_manager.cc:194] Registered new tserver with Master: b7c3f8a765fb487393c28d9f9db02e93 (127.10.242.2:34913)
I20260810 06:44:38.933827 24826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:43403
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:44:38.971101 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34183
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:39.294049 25388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:39.294375 25388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:39.294519 25388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:39.301548 25388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:39.301668 25388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:39.301815 25388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:39.311358 25388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:34183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:39.312597 25388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:39.313897 25388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:39.327354 25398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:39.327323 25401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:39.327678 25404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:39.328464 25388 server_base.cc:1034] running on GCE node
I20260810 06:44:39.329059 25388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:39.329931 25388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:39.331127 25388 hybrid_clock.cc:648] HybridClock initialized: now 1786344279331084 us; error 27 us; skew 500 ppm
I20260810 06:44:39.331435 25388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:39 dist-test-slave-zj8q krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344279, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:39.334396 25388 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:44:39.335825 25388 webserver.cc:460] Webserver started at http://127.10.242.3:38625/ using document root <none> and password file <none>
I20260810 06:44:39.336476 25388 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:39.336591 25388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:39.336884 25388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:39.339800 25388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "6a687bbd73b744c2b53c33d6f596938d"
format_stamp: "Formatted at 2026-08-10 06:44:39 on dist-test-slave-zj8q"
server_key: "832183efda01da134e50fc799303170d"
server_key_iv: "9bc54ddff4787faec21aabab9f9ef5c3"
server_key_version: "encryptionkey@0"
I20260810 06:44:39.340550 25388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "6a687bbd73b744c2b53c33d6f596938d"
format_stamp: "Formatted at 2026-08-10 06:44:39 on dist-test-slave-zj8q"
server_key: "832183efda01da134e50fc799303170d"
server_key_iv: "9bc54ddff4787faec21aabab9f9ef5c3"
server_key_version: "encryptionkey@0"
I20260810 06:44:39.345371 25388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 06:44:39.348631 25414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:39.349864 25388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:44:39.350013 25388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "6a687bbd73b744c2b53c33d6f596938d"
format_stamp: "Formatted at 2026-08-10 06:44:39 on dist-test-slave-zj8q"
server_key: "832183efda01da134e50fc799303170d"
server_key_iv: "9bc54ddff4787faec21aabab9f9ef5c3"
server_key_version: "encryptionkey@0"
I20260810 06:44:39.350229 25388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:39.364586 25388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:39.368229 25388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:39.368515 25388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:39.369417 25388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:39.370679 25388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:39.370751 25388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:39.370836 25388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:39.370882 25388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:39.400439 25388 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:40917
I20260810 06:44:39.400489 25600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:40917 every 8 connection(s)
I20260810 06:44:39.401674 25388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:39.403506 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 25388
I20260810 06:44:39.403683 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:44:39.404131 11208 external_mini_cluster.cc:1428] Setting key a90ba9c5f02bf039647ad653b9293d27
Aug 10 06:44:39 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344279, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:44:39.421627 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:39.404236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:36263 (local address 127.10.242.62:34183)
0810 06:44:39.404434 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:39.404441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:39.405760 (+ 1319us) server_negotiation.cc:407] Connection header received
0810 06:44:39.407985 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:39.407989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:39.408071 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:39.408154 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:39.409598 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.410397 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.411847 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.412254 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.412343 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:39.416245 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:39.416249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:39.416253 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:39.416305 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:39.419206 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:39.419872 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:39.419876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:39.419878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:39.420007 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:39.420337 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:39.420340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:39.420342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:39.420593 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:39.420711 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:39.421244 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 06:44:39.421400 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:44:39.422561 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:39.404773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:36263)
0810 06:44:39.405590 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:39.405648 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:39.407566 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:39.408347 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:39.408354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:39.408892 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:39.409440 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:39.409457 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.410511 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.410515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:39.411718 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:39.411730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.412414 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.412418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:39.412687 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:39.413379 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:39.413407 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:39.416045 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:39.419347 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:39.419356 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:39.419369 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:39.419778 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:39.420088 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:44:39.420091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:44:39.420093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:44:39.420263 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:44:39.420682 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:39.420687 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:39.420952 (+ 265us) client_negotiation.cc:769] Sending connection context
0810 06:44:39.421169 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 06:44:39.421494 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":314,"threads_started":1}
I20260810 06:44:39.423748 25602 heartbeater.cc:347] Connected to a master server at 127.10.242.62:34183
I20260810 06:44:39.424019 25602 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:39.424703 25602 heartbeater.cc:511] Master 127.10.242.62:34183 requested a full tablet report, sending...
I20260810 06:44:39.425985 24826 ts_manager.cc:194] Registered new tserver with Master: 6a687bbd73b744c2b53c33d6f596938d (127.10.242.3:40917)
I20260810 06:44:39.426800 24826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:36263
I20260810 06:44:39.431280 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:44:39.452862 25144 heartbeater.cc:503] Master 127.10.242.62:34183 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab.
Entry for principal oryx/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab.
WARNING: no policy specified for oryx/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab.
Entry for principal oryx/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab.
WARNING: no policy specified for oryx/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab.
Entry for principal oryx/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab.
I20260810 06:44:39.569854 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 24929
Aug 10 06:44:39 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:44:39.585845 25616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:39.571076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.10.242.1:42353)
0810 06:44:39.571982 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:39.571989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:39.572015 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:44:39.572078 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:39.572084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:39.572307 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:39.572442 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:39.573792 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.574859 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.576046 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.576602 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.576697 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:39.580527 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:39.580538 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:39.580553 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:39.580586 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:39.583166 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:39.583791 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:39.583799 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:39.583801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:39.583933 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:39.584301 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:39.584304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:39.584306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:39.584634 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:39.584932 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:39.585223 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 06:44:39.585481 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":616,"threads_started":1}
I20260810 06:44:39.587105 25134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:39.668690 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 24929
I20260810 06:44:39.694416 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 25156
Aug 10 06:44:39 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:44:39.709225 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:39.695310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57422 (local address 127.10.242.2:34913)
0810 06:44:39.695883 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:39.695890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:39.695917 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:44:39.695981 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:39.695987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:39.696211 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:39.696340 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:39.697531 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.698578 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.699822 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.700370 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.700460 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:39.704446 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:39.704457 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:39.704472 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:39.704504 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:39.707041 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:39.707615 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:39.707620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:39.707623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:39.707736 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:39.708001 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:39.708004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:39.708005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:39.708326 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:39.708559 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:39.708749 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 06:44:39.708939 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":292,"threads_started":1}
I20260810 06:44:39.710513 25371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:39.790177 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 25156
I20260810 06:44:39.812405 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 25388
Aug 10 06:44:39 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:44:39.827556 25621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:39.813275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.10.242.3:40917)
0810 06:44:39.813907 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:39.813915 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:39.813945 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:44:39.814006 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:39.814012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:39.814263 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:39.814395 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:39.815615 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.816635 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.817786 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:39.818487 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:39.818588 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:39.822625 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:39.822637 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:39.822653 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:39.822686 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:39.825185 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:39.825822 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:39.825829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:39.825831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:39.825946 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:39.826248 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:39.826251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:39.826253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:39.826578 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:39.826824 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:39.827035 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 06:44:39.827263 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":298,"threads_started":1}
I20260810 06:44:39.828886 25593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:39.909624 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 25388
I20260810 06:44:39.932224 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 24768
I20260810 06:44:39.933410 24880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:40.037245 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 24768
I20260810 06:44:40.063776 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34183
--webserver_interface=127.10.242.62
--webserver_port=46577
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:34183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:44:40.392005 25624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:40.392304 25624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:40.392441 25624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:40.399582 25624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:40.399690 25624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:40.399768 25624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:40.399844 25624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:40.399902 25624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:40.409473 25624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:34183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=46577
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:40.410775 25624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:40.412051 25624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:40.423431 25631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:40.423663 25635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:40.423683 25632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:40.424463 25624 server_base.cc:1034] running on GCE node
I20260810 06:44:40.425208 25624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:40.426578 25624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:40.427783 25624 hybrid_clock.cc:648] HybridClock initialized: now 1786344280427730 us; error 38 us; skew 500 ppm
I20260810 06:44:40.428110 25624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:40 dist-test-slave-zj8q krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344280, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:40.431083 25624 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:44:40.432488 25624 webserver.cc:460] Webserver started at http://127.10.242.62:46577/ using document root <none> and password file <none>
I20260810 06:44:40.433137 25624 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:40.433267 25624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:40.438755 25624 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:44:40.441461 25645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:40.442709 25624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:40.442863 25624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "623b0c8909f64a6d8451e670b171063e"
format_stamp: "Formatted at 2026-08-10 06:44:37 on dist-test-slave-zj8q"
server_key: "1144231fe420e1588cfba541faa95378"
server_key_iv: "f95752c0c25a07996005435742ca034d"
server_key_version: "encryptionkey@0"
I20260810 06:44:40.443789 25624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:40.463236 25624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:40.466938 25624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:40.467186 25624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:40.484192 25624 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:34183
I20260810 06:44:40.484228 25734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:34183 every 8 connection(s)
I20260810 06:44:40.485637 25624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:40.486786 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 25624
I20260810 06:44:40.487550 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:42353
--local_ip_for_outbound_sockets=127.10.242.1
--tserver_master_addrs=127.10.242.62:34183
--webserver_port=34855
--webserver_interface=127.10.242.1
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.242.1
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:44:40.492424 25735 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:44:40.495589 25735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: Bootstrap starting.
I20260810 06:44:40.513841 25735 log.cc:824] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:40.522589 25735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:44:40.523085 25735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: Bootstrap complete.
I20260810 06:44:40.531838 25735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } }
I20260810 06:44:40.532646 25735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 623b0c8909f64a6d8451e670b171063e, State: Initialized, Role: FOLLOWER
I20260810 06:44:40.533226 25735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } }
I20260810 06:44:40.533444 25735 raft_consensus.cc:388] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:40.533555 25735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:40.533720 25735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:44:40.537405 25735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } }
I20260810 06:44:40.537863 25735 leader_election.cc:302] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 623b0c8909f64a6d8451e670b171063e; no voters:
I20260810 06:44:40.538470 25735 leader_election.cc:288] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:44:40.538611 25741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:44:40.540588 25735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:40.540594 25741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [term 2 LEADER]: Becoming Leader. State: Replica: 623b0c8909f64a6d8451e670b171063e, State: Running, Role: LEADER
I20260810 06:44:40.541215 25741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } }
I20260810 06:44:40.544284 25742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "623b0c8909f64a6d8451e670b171063e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } } }
I20260810 06:44:40.544322 25745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 623b0c8909f64a6d8451e670b171063e. Latest consensus state: current_term: 2 leader_uuid: "623b0c8909f64a6d8451e670b171063e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "623b0c8909f64a6d8451e670b171063e" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34183 } } }
I20260810 06:44:40.544693 25742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:40.544697 25745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:40.555071 25758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:40.561133 25758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:40.562608 25758 catalog_manager.cc:1259] Loaded cluster ID: 3bb22910a7524026bcdd0772a86e441e
I20260810 06:44:40.562796 25758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:40.566866 25758 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:40.568702 25758 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 623b0c8909f64a6d8451e670b171063e: Loaded TSK: 0
I20260810 06:44:40.570361 25758 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:44:40.828403 25738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:40.828688 25738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:40.828826 25738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:40.836021 25738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:40.836148 25738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:40.836292 25738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:44:40.846453 25738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.1.keytab
--principal=oryx/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:42353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=34855
--tserver_master_addrs=127.10.242.62:34183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:40.847781 25738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:40.849069 25738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:40.863193 25786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:40.863404 25790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:40.863864 25784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:40.864363 25738 server_base.cc:1034] running on GCE node
I20260810 06:44:40.864950 25738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:40.865887 25738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:40.867106 25738 hybrid_clock.cc:648] HybridClock initialized: now 1786344280867060 us; error 38 us; skew 500 ppm
I20260810 06:44:40.867399 25738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:40 dist-test-slave-zj8q krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344280, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:40.870469 25738 init.cc:377] Logged in from keytab as oryx/127.10.242.1@KRBTEST.COM (short username oryx)
I20260810 06:44:40.871920 25738 webserver.cc:460] Webserver started at http://127.10.242.1:34855/ using document root <none> and password file <none>
I20260810 06:44:40.872576 25738 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:40.872713 25738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:40.877921 25738 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:44:40.880575 25800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:40.881898 25738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:44:40.882053 25738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "d0aeed5737e94cd7abd6c97197ce397a"
format_stamp: "Formatted at 2026-08-10 06:44:38 on dist-test-slave-zj8q"
server_key: "372db909affc3b307336a82245276ba9"
server_key_iv: "e211889cf397d189b415f21fc70da394"
server_key_version: "encryptionkey@0"
I20260810 06:44:40.883049 25738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:40.897894 25738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:40.901510 25738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:40.901759 25738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:40.902740 25738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:40.904017 25738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:40.904112 25738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:40.904215 25738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:40.904294 25738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:40.934319 25738 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:42353
I20260810 06:44:40.934379 26002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:42353 every 8 connection(s)
I20260810 06:44:40.935708 25738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:40.943938 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 25738
I20260810 06:44:40.944528 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:34913
--local_ip_for_outbound_sockets=127.10.242.2
--tserver_master_addrs=127.10.242.62:34183
--webserver_port=38777
--webserver_interface=127.10.242.2
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.242.2
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:44:40 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:40 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:44:41.276371 26012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:41.276640 26012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:41.276773 26012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:41.283849 26012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:41.283959 26012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:41.284102 26012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:41.293730 26012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.2.keytab
--principal=oryx/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:34913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=38777
--tserver_master_addrs=127.10.242.62:34183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:41.295194 26012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:41.296455 26012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:41.312270 26026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:41.312527 26029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:41.312772 26022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:41.313227 26012 server_base.cc:1034] running on GCE node
I20260810 06:44:41.313841 26012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:41.314849 26012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:41.316051 26012 hybrid_clock.cc:648] HybridClock initialized: now 1786344281315986 us; error 61 us; skew 500 ppm
I20260810 06:44:41.316357 26012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:41 dist-test-slave-zj8q krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344281, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:41.319424 26012 init.cc:377] Logged in from keytab as oryx/127.10.242.2@KRBTEST.COM (short username oryx)
I20260810 06:44:41.320955 26012 webserver.cc:460] Webserver started at http://127.10.242.2:38777/ using document root <none> and password file <none>
I20260810 06:44:41.321617 26012 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:41.321744 26012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:41.327023 26012 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260810 06:44:41.329759 26045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:41.331580 26012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:44:41.331751 26012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "b7c3f8a765fb487393c28d9f9db02e93"
format_stamp: "Formatted at 2026-08-10 06:44:38 on dist-test-slave-zj8q"
server_key: "0d709e0862335f6e6559c39481028fd0"
server_key_iv: "03c332250c2d3a62cebf55f8fa806a81"
server_key_version: "encryptionkey@0"
I20260810 06:44:41.333047 26012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:41.364214 26012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:41.368242 26012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:41.368495 26012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:41.369522 26012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:41.370942 26012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:41.371043 26012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:41.371147 26012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:41.371217 26012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:41.411267 26012 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:34913
I20260810 06:44:41.411322 26233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:34913 every 8 connection(s)
I20260810 06:44:41.412734 26012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:41.413784 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 26012
I20260810 06:44:41.414784 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:40917
--local_ip_for_outbound_sockets=127.10.242.3
--tserver_master_addrs=127.10.242.62:34183
--webserver_port=38625
--webserver_interface=127.10.242.3
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:46735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.242.3
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:44:41 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:41 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:44:41.762840 26240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:41.763124 26240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:41.763283 26240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:41.770519 26240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:41.770640 26240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:41.770798 26240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:41.780349 26240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:46735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/krb5kdc/oryx_127.10.242.3.keytab
--principal=oryx/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:40917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=38625
--tserver_master_addrs=127.10.242.62:34183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:41.781627 26240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:41.782925 26240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:41.798430 26247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:41.798496 26248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:41.802639 26255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:41.805639 26240 server_base.cc:1034] running on GCE node
I20260810 06:44:41.806283 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:41.807381 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:41.808676 26240 hybrid_clock.cc:648] HybridClock initialized: now 1786344281808628 us; error 42 us; skew 500 ppm
I20260810 06:44:41.808969 26240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:44:41 dist-test-slave-zj8q krb5kdc[24747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344281, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:44:41.812011 26240 init.cc:377] Logged in from keytab as oryx/127.10.242.3@KRBTEST.COM (short username oryx)
I20260810 06:44:41.813622 26240 webserver.cc:460] Webserver started at http://127.10.242.3:38625/ using document root <none> and password file <none>
I20260810 06:44:41.814324 26240 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:41.814460 26240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:41.819598 26240 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260810 06:44:41.822283 26268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:41.823599 26240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:44:41.823752 26240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "6a687bbd73b744c2b53c33d6f596938d"
format_stamp: "Formatted at 2026-08-10 06:44:39 on dist-test-slave-zj8q"
server_key: "832183efda01da134e50fc799303170d"
server_key_iv: "9bc54ddff4787faec21aabab9f9ef5c3"
server_key_version: "encryptionkey@0"
I20260810 06:44:41.824667 26240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:41.844504 26240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:41.848799 26240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:41.849040 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:41.850035 26240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:41.851456 26240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:41.851552 26240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:41.851668 26240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:41.851739 26240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:41.890100 26240 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:40917
I20260810 06:44:41.890317 26484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:40917 every 8 connection(s)
I20260810 06:44:41.891788 26240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:41.899087 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 26240
Aug 10 06:44:41 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:41 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:41.923123 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:41.900130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48956 (local address 127.10.242.62:34183)
0810 06:44:41.904761 (+ 4631us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:41.904766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:41.904782 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:44:41.904834 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:41.904837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:41.904918 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:41.905024 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:41.906927 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.908152 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.909390 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.909994 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.910127 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:41.913449 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:41.913460 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:41.913480 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:41.913530 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:41.917468 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:41.918617 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:41.918627 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:41.918635 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:41.918834 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:44:41.919350 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:44:41.919355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:44:41.919357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:44:41.919964 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:41.921376 (+ 1412us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:41.921658 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 06:44:41.922100 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4515,"thread_start_us":4422,"threads_started":1}
W20260810 06:44:42.239797 26010 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.291s user 0.000s sys 0.014s
W20260810 06:44:42.241725 26011 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:40.938472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:34919 (local address 127.10.242.62:34183)
0810 06:44:40.939330 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:40.939345 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:40.939530 (+ 185us) server_negotiation.cc:407] Connection header received
0810 06:44:40.941753 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:40.941765 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:40.942055 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:40.942410 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:40.943880 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:40.945182 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:40.946730 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:40.947408 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:40.947575 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.241329 (+1293754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:34919: BlockingRecv error: recv got EOF from 127.10.242.1:34919 (error 108)
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":351,"threads_started":1}
I20260810 06:44:42.242596 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:40.938510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:34919)
0810 06:44:40.939361 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:40.939417 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:40.941207 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:40.942581 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:40.942592 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:40.943180 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:40.943699 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:40.943716 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:40.945335 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:40.945339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:40.946597 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:40.946609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:40.947628 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:40.947635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:40.947887 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:40.948534 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:40.948557 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:42.240964 (+1292407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":333,"threads_started":1}
W20260810 06:44:42.243881 26003 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:34183 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.242.62:34183: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:42.258191 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:42.245027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:33445)
0810 06:44:42.245278 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:42.245304 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:42.245446 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:42.246169 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:42.246175 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:42.246721 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:42.246979 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.246993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.248674 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.248680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.250431 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.250449 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.251490 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.251497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.251698 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.252504 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:42.252532 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:42.257797 (+ 5265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260810 06:44:42.258841 26011 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:42.245470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:33445 (local address 127.10.242.62:34183)
0810 06:44:42.245781 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:42.245788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:42.245808 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:44:42.245870 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:42.245876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:42.245960 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:42.246081 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:42.247158 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.248517 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.250596 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.251300 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.251431 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.258106 (+ 6675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:33445: BlockingRecv error: recv got EOF from 127.10.242.1:33445 (error 108)
Metrics: {"server-negotiator.queue_time_us":182}
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:42.272854 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:42.259183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:56071)
0810 06:44:42.260736 (+ 1553us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:42.260760 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:42.260939 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:42.261561 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:42.261566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:42.262016 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:42.262269 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.262281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.263833 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.263838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.265526 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.265543 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.266643 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.266650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.266851 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.267566 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:42.267592 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:42.272537 (+ 4945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536}
W20260810 06:44:42.273195 26495 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:42.259408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:56071 (local address 127.10.242.62:34183)
0810 06:44:42.259650 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:42.259656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:42.260835 (+ 1179us) server_negotiation.cc:407] Connection header received
0810 06:44:42.261088 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:42.261094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:42.261182 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:42.261254 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:42.262452 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.263681 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.265703 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.266483 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.266627 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.272875 (+ 6248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:56071: BlockingRecv error: recv got EOF from 127.10.242.1:56071 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 06:44:42.273614 26003 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:44:42.600412 26238 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.160s user 0.008s sys 0.008s
W20260810 06:44:42.602003 26237 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:41.416303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:53875 (local address 127.10.242.62:34183)
0810 06:44:41.431963 (+ 15660us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:41.431971 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:41.431998 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:44:41.432087 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:41.432093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:41.432184 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:41.432325 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:41.434374 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.435633 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.437965 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.438733 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.438975 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.601755 (+1162780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:53875: BlockingRecv error: recv got EOF from 127.10.242.2:53875 (error 108)
Metrics: {"server-negotiator.queue_time_us":15505,"thread_start_us":15341,"threads_started":1}
I20260810 06:44:42.603096 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:41.416858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:53875)
0810 06:44:41.428770 (+ 11912us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:41.428818 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:41.431411 (+ 2593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:41.432601 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:41.432612 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:41.433309 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:41.434193 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:41.434219 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.435810 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.435815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:41.437811 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:41.437826 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.439067 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.439075 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:41.439346 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:41.440184 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:41.440216 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:42.601581 (+1161365us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":11579,"thread_start_us":11355,"threads_started":1}
W20260810 06:44:42.604004 26234 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:34183 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.242.62:34183: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:44:42.614307 26237 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:42.604732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:42313 (local address 127.10.242.62:34183)
0810 06:44:42.604872 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:42.604877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:42.604927 (+ 50us) server_negotiation.cc:407] Connection header received
0810 06:44:42.605115 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:42.605119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:42.605200 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:42.605283 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:42.606224 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.607194 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.608429 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.609002 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.609114 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.614087 (+ 4973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:42313: BlockingRecv error: recv got EOF from 127.10.242.2:42313 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:44:42.614308 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:42.604613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:42313)
0810 06:44:42.604843 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:42.604867 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:42.605011 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:42.605342 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:42.605346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:42.605783 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:42.605995 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.606009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.607335 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.607341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.608292 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.608304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.609180 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.609187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.609348 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.609908 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:42.609926 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:42.613933 (+ 4007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:42.625337 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:42.614965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:56973)
0810 06:44:42.615115 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:42.615130 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:42.615249 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:42.615753 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:42.615756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:42.616019 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:42.616190 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.616198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.617772 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.617778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.619496 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.619513 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.620188 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.620194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.620402 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.620963 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:42.620988 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:42.625082 (+ 4094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 06:44:42.625531 26237 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:42.615243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:56973 (local address 127.10.242.62:34183)
0810 06:44:42.615430 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:42.615436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:42.615455 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:44:42.615511 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:42.615516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:42.615607 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:42.615720 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:42.616323 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.617558 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.619638 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.620046 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.620137 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.625282 (+ 5145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:56973: BlockingRecv error: recv got EOF from 127.10.242.2:56973 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:44:42.625675 26234 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:44:42.935214 26493 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.017s user 0.000s sys 0.014s
W20260810 06:44:42.936560 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:41.894603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:42565 (local address 127.10.242.62:34183)
0810 06:44:41.902534 (+ 7931us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:41.902540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:41.905924 (+ 3384us) server_negotiation.cc:407] Connection header received
0810 06:44:41.905998 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:41.906003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:41.906089 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:41.906262 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:41.910869 (+ 4607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.912013 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.915770 (+ 3757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.916526 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.916621 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.936297 (+1019676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:42565: BlockingRecv error: recv got EOF from 127.10.242.3:42565 (error 108)
Metrics: {"server-negotiator.queue_time_us":7797,"thread_start_us":7667,"threads_started":1}
I20260810 06:44:42.937284 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:41.895291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:42565)
0810 06:44:41.902749 (+ 7458us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:41.902807 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:41.905502 (+ 2695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:41.907402 (+ 1900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:41.907415 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:41.908121 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:41.910671 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:41.910699 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.912237 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.912243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:41.915600 (+ 3357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:41.915617 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:41.917007 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:41.917015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:41.917324 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:41.917931 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:41.917967 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:42.936084 (+1018117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":7092,"thread_start_us":6960,"threads_started":1}
W20260810 06:44:42.938220 26485 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:34183 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.242.62:34183: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:42.951370 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:42.938955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:47971)
0810 06:44:42.939146 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:42.939167 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:42.939280 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:42.939538 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:42.939542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:42.939874 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:42.940036 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.940060 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.941109 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.941114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.942139 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.942180 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.942832 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.942836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.942953 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.947758 (+ 4805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:42.947777 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:42.951164 (+ 3387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 06:44:42.951519 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:42.939056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:47971 (local address 127.10.242.62:34183)
0810 06:44:42.939260 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:42.939264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:42.939282 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:44:42.939371 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:42.939375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:42.939432 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:42.939504 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:42.940154 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.940971 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.942282 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.942688 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.942793 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.951341 (+ 8548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:47971: BlockingRecv error: recv got EOF from 127.10.242.3:47971 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:42 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:42.958988 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:42.952019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:37259)
0810 06:44:42.952168 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:42.952186 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:42.952278 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:42.952518 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:42.952522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:42.952793 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:42.952936 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.952943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.953803 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.953806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.954795 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:42.954810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.955432 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.955438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:42.955572 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.956053 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:42.956069 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:42.958813 (+ 2744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:44:42.959208 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:42.952118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:37259 (local address 127.10.242.62:34183)
0810 06:44:42.952310 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:42.952314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:42.952328 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:44:42.952365 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:42.952368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:42.952418 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:42.952487 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:42.953035 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.953686 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.954903 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:42.955310 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:42.955422 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:42.958978 (+ 3556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:37259: BlockingRecv error: recv got EOF from 127.10.242.3:37259 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260810 06:44:42.959242 26485 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 06:44:43 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:43 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:43.283034 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:43.274215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:38359)
0810 06:44:43.274755 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:43.274771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:43.274890 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:43.275180 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:43.275183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:43.275609 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:43.275803 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:43.275814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.277038 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.277042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:43.278067 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:43.278081 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.278854 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.278860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:43.279007 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:43.279513 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:43.279530 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:43.282791 (+ 3261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":298,"threads_started":1}
W20260810 06:44:43.283181 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:43.274268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:38359 (local address 127.10.242.62:34183)
0810 06:44:43.274434 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:43.274439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:43.274842 (+ 403us) server_negotiation.cc:407] Connection header received
0810 06:44:43.274980 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:43.274983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:43.275063 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:43.275143 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:43.275947 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.276854 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.278229 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.278670 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.278781 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:43.282978 (+ 4197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:38359: BlockingRecv error: recv got EOF from 127.10.242.1:38359 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 06:44:43 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:43 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:43.635076 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:43.626227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:43329)
0810 06:44:43.626785 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:43.626806 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:43.626950 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:43.627314 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:43.627319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:43.627841 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:43.628045 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:43.628081 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.629156 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.629162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:43.630121 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:43.630133 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.630900 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.630906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:43.631084 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:43.631599 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:43.631617 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:43.634849 (+ 3232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":296,"threads_started":1}
W20260810 06:44:43.635277 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:43.626287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:43329 (local address 127.10.242.62:34183)
0810 06:44:43.626465 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:43.626478 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:43.626895 (+ 417us) server_negotiation.cc:407] Connection header received
0810 06:44:43.627064 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:43.627069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:43.627136 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:43.627263 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:43.628232 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.628992 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.630283 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.630724 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.630839 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:43.635043 (+ 4204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:43329: BlockingRecv error: recv got EOF from 127.10.242.2:43329 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 06:44:43 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:43 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:43.968546 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:43.959794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:34733)
0810 06:44:43.960322 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:43.960338 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:43.960468 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:43.960809 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:43.960813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:43.961262 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:43.961414 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:43.961423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.962682 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.962688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:43.963707 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:43.963720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.964428 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.964432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:43.964553 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:43.965106 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:43.965124 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:43.968315 (+ 3191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":288,"threads_started":1}
W20260810 06:44:43.968698 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:43.959963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:34733 (local address 127.10.242.62:34183)
0810 06:44:43.960182 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:43.960188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:43.960415 (+ 227us) server_negotiation.cc:407] Connection header received
0810 06:44:43.960578 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:43.960584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:43.960671 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:43.960755 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:43.961551 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.962517 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.963828 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:43.964288 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:43.964381 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:43.968503 (+ 4122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:34733: BlockingRecv error: recv got EOF from 127.10.242.3:34733 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 06:44:44 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:44 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:44.292654 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:44.283950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:35773)
0810 06:44:44.284419 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:44.284435 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:44.284552 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:44.284828 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:44.284831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:44.285239 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:44.285430 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:44.285440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.286637 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.286641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:44.287829 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:44.287842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.288582 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.288586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:44.288712 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:44.289226 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:44.289242 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:44.292392 (+ 3150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":257,"threads_started":1}
W20260810 06:44:44.292855 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:44.284000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:35773 (local address 127.10.242.62:34183)
0810 06:44:44.284225 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:44.284232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:44.284509 (+ 277us) server_negotiation.cc:407] Connection header received
0810 06:44:44.284657 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:44.284660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:44.284724 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:44.284793 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:44.285555 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.286497 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.287948 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.288454 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.288550 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:44.292609 (+ 4059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:35773: BlockingRecv error: recv got EOF from 127.10.242.1:35773 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 06:44:44 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:44 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:44.644393 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:44.635878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:48599)
0810 06:44:44.636348 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:44.636364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:44.636492 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:44.636761 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:44.636764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:44.637200 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:44.637405 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:44.637418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.638493 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.638497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:44.639489 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:44.639502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.640136 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.640140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:44.640263 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:44.640831 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:44.640850 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:44.644159 (+ 3309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":256,"threads_started":1}
W20260810 06:44:44.644550 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:44.636013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:48599 (local address 127.10.242.62:34183)
0810 06:44:44.636273 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:44.636280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:44.636432 (+ 152us) server_negotiation.cc:407] Connection header received
0810 06:44:44.636588 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:44.636592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:44.636652 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:44.636729 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:44.637527 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.638362 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.639612 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.640023 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.640119 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:44.644345 (+ 4226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:48599: BlockingRecv error: recv got EOF from 127.10.242.2:48599 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 10 06:44:44 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:44 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:44.977941 26515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:44.969402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:36837)
0810 06:44:44.969889 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:44.969904 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:44.970045 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:44.970385 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:44.970390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:44.970817 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:44.971016 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:44.971025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.972038 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.972043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:44.972993 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:44.973005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.973701 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.973706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:44.973876 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:44.974404 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:44.974421 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:44.977698 (+ 3277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":257,"threads_started":1}
W20260810 06:44:44.978107 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:44.969594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:36837 (local address 127.10.242.62:34183)
0810 06:44:44.969823 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:44.969830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:44.969981 (+ 151us) server_negotiation.cc:407] Connection header received
0810 06:44:44.970184 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:44.970190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:44.970258 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:44.970337 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:44.971159 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.971911 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.973133 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:44.973548 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:44.973655 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:44.977894 (+ 4239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:36837: BlockingRecv error: recv got EOF from 127.10.242.3:36837 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 10 06:44:45 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:45 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:45.302067 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:45.293504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:42239)
0810 06:44:45.294019 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:45.294034 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:45.294182 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:45.294489 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:45.294494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:45.294883 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:45.295053 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:45.295063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.296099 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.296103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:45.297028 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:45.297041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.297776 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.297782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:45.297967 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:45.298582 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:45.298600 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:45.301817 (+ 3217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":290,"threads_started":1}
W20260810 06:44:45.302284 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:45.293602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:42239 (local address 127.10.242.62:34183)
0810 06:44:45.293810 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:45.293817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:45.294115 (+ 298us) server_negotiation.cc:407] Connection header received
0810 06:44:45.294296 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:45.294300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:45.294360 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:45.294438 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:45.295220 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.295950 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.297167 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.297615 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.297724 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:45.302023 (+ 4299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:42239: BlockingRecv error: recv got EOF from 127.10.242.1:42239 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 06:44:45 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:45 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:45.653755 26517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:45.645252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:53859)
0810 06:44:45.645780 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:45.645796 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:45.645928 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:45.646282 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:45.646287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:45.646709 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:45.646907 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:45.646919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.647980 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.647985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:45.648897 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:45.648911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.649600 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.649606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:45.649791 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:45.650375 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:45.650393 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:45.653521 (+ 3128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":276,"threads_started":1}
W20260810 06:44:45.653963 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:45.645309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:53859 (local address 127.10.242.62:34183)
0810 06:44:45.645471 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:45.645475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:45.645878 (+ 403us) server_negotiation.cc:407] Connection header received
0810 06:44:45.646051 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:45.646055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:45.646116 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:45.646247 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:45.647061 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.647831 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.649032 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.649463 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.649570 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:45.653745 (+ 4175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:53859: BlockingRecv error: recv got EOF from 127.10.242.2:53859 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 06:44:45 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:45 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:45.987764 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:45.978803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:36397)
0810 06:44:45.979537 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:45.979556 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:45.979684 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:45.979966 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:45.979969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:45.980353 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:45.980535 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:45.980543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.981500 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.981504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:45.982430 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:45.982442 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.983204 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.983210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:45.983396 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:45.984039 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:45.984061 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:45.987510 (+ 3449us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":488,"threads_started":1}
W20260810 06:44:45.987898 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:45.978873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:36397 (local address 127.10.242.62:34183)
0810 06:44:45.979141 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:45.979149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:45.979639 (+ 490us) server_negotiation.cc:407] Connection header received
0810 06:44:45.979789 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:45.979793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:45.979857 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:45.979927 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:45.980647 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.981394 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.982530 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:45.983048 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:45.983168 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:45.987701 (+ 4533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:36397: BlockingRecv error: recv got EOF from 127.10.242.3:36397 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 10 06:44:46 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:46 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:46.311796 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:46.302973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:32899)
0810 06:44:46.303545 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:46.303562 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:46.303697 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:46.303969 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:46.303973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:46.304353 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:46.304521 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:46.304531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.305465 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.305468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:46.306520 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:46.306531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.307113 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.307117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:46.307240 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:46.307774 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:46.307792 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:46.311442 (+ 3650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":323,"threads_started":1}
W20260810 06:44:46.311909 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:46.303047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:32899 (local address 127.10.242.62:34183)
0810 06:44:46.303235 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:46.303240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:46.303629 (+ 389us) server_negotiation.cc:407] Connection header received
0810 06:44:46.303785 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:46.303788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:46.303846 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:46.303923 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:46.304629 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.305369 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.306617 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.307008 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.307095 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:46.311676 (+ 4581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:32899: BlockingRecv error: recv got EOF from 127.10.242.1:32899 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 06:44:46 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:46 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:46.663751 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:46.654652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:39817)
0810 06:44:46.655185 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:46.655203 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:46.655335 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:46.655640 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:46.655644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:46.656110 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:46.656306 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:46.656320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.657378 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.657383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:46.658793 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:46.658808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.659481 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.659484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:46.659604 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:46.660246 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:46.660265 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:46.663399 (+ 3134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":281,"threads_started":1}
W20260810 06:44:46.663826 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:46.654757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:39817 (local address 127.10.242.62:34183)
0810 06:44:46.654947 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:46.654954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:46.655270 (+ 316us) server_negotiation.cc:407] Connection header received
0810 06:44:46.655445 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:46.655450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:46.655509 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:46.655587 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:46.656448 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.657248 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.658925 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.659374 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.659469 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:46.663607 (+ 4138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:39817: BlockingRecv error: recv got EOF from 127.10.242.2:39817 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 06:44:46 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:46 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:46.997236 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:46.988648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:46109)
0810 06:44:46.989109 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:46.989133 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:46.989245 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:46.989526 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:46.989529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:46.989899 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:46.990077 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:46.990085 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.991075 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.991078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:46.992118 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:46.992140 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.992759 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.992762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:46.992878 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:46.993407 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:46.993424 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:46.996904 (+ 3480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":235,"threads_started":1}
W20260810 06:44:46.997390 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:46.988714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:46109 (local address 127.10.242.62:34183)
0810 06:44:46.988926 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:46.988932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:46.989209 (+ 277us) server_negotiation.cc:407] Connection header received
0810 06:44:46.989348 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:46.989352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:46.989413 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:46.989491 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:46.990229 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.990958 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.992252 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:46.992654 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:46.992743 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:46.997117 (+ 4374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:46109: BlockingRecv error: recv got EOF from 127.10.242.3:46109 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 06:44:47 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:47 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:47.321688 26523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:47.312757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:51373)
0810 06:44:47.313241 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:47.313257 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:47.313391 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:47.313762 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:47.313767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:47.314229 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:47.314393 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:47.314403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:47.315420 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:47.315425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:47.316433 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:47.316447 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:47.317122 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:47.317128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:47.317308 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:47.317992 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:47.318015 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:47.321353 (+ 3338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":248,"threads_started":1}
W20260810 06:44:47.321821 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:47.312834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:51373 (local address 127.10.242.62:34183)
0810 06:44:47.313037 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:47.313044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:47.313343 (+ 299us) server_negotiation.cc:407] Connection header received
0810 06:44:47.313509 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:47.313513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:47.313575 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:47.313656 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:47.314540 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:47.315287 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:47.316574 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:47.316982 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:47.317091 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:47.321577 (+ 4486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:51373: BlockingRecv error: recv got EOF from 127.10.242.1:51373 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 06:44:47 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:47 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:47.673395 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:47.664704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:50419)
0810 06:44:47.665209 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:47.665224 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:47.665353 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:47.665700 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:47.665704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:47.666119 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:47.666307 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:47.666317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:47.667354 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:47.667359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:47.668298 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:47.668311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:47.669020 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:47.669026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:47.669197 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:47.669775 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:47.669793 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:47.673081 (+ 3288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":253,"threads_started":1}
W20260810 06:44:47.673545 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:47.664861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:50419 (local address 127.10.242.62:34183)
0810 06:44:47.665060 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:47.665065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:47.665304 (+ 239us) server_negotiation.cc:407] Connection header received
0810 06:44:47.665466 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:47.665471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:47.665551 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:47.665631 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:47.666448 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:47.667216 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:47.668436 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:47.668879 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:47.668989 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:47.673313 (+ 4324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:50419: BlockingRecv error: recv got EOF from 127.10.242.2:50419 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 06:44:48 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:48 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:48.012331 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:47.998387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:49063)
0810 06:44:47.998885 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:47.998901 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:47.999047 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:47.999449 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:47.999455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:47.999966 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:48.000150 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:48.000160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.001148 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.001152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:48.002116 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:48.002128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.002956 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.002960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:48.003102 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:48.008341 (+ 5239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:48.008359 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:48.012088 (+ 3729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":255,"threads_started":1}
W20260810 06:44:48.012471 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:47.998444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:49063 (local address 127.10.242.62:34183)
0810 06:44:47.998637 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:47.998643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:47.998982 (+ 339us) server_negotiation.cc:407] Connection header received
0810 06:44:47.999170 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:47.999175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:47.999262 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:47.999358 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:48.000285 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.001021 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.002259 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.002833 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.002929 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:48.012269 (+ 9340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:49063: BlockingRecv error: recv got EOF from 127.10.242.3:49063 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:44:48 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:48 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:48.331362 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:48.322646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:42191)
0810 06:44:48.323142 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:48.323160 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:48.323278 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:48.323591 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:48.323595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:48.324142 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:48.324392 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:48.324405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.325460 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.325464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:48.326469 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:48.326482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.327115 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.327120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:48.327245 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:48.327798 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:48.327821 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:48.331131 (+ 3310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":273,"threads_started":1}
W20260810 06:44:48.331583 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:48.322711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:42191 (local address 127.10.242.62:34183)
0810 06:44:48.322902 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:48.322906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:48.323231 (+ 325us) server_negotiation.cc:407] Connection header received
0810 06:44:48.323379 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:48.323383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:48.323445 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:48.323540 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:48.324532 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.325331 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.326574 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.326993 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.327089 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:48.331354 (+ 4265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:42191: BlockingRecv error: recv got EOF from 127.10.242.1:42191 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 06:44:48 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:48 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:48.682435 26532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:48.674292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:59825)
0810 06:44:48.674736 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:48.674752 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:48.674857 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:48.675093 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:48.675096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:48.675501 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:48.675666 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:48.675676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.676611 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.676615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:48.677583 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:48.677596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.678257 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.678261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:48.678382 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:48.678885 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:48.678902 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:48.682094 (+ 3192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":218,"threads_started":1}
W20260810 06:44:48.682578 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:48.674701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:59825 (local address 127.10.242.62:34183)
0810 06:44:48.674853 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:48.674857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:48.674873 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:44:48.674938 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:48.674941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:48.675000 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:48.675067 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:48.675776 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.676478 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.677687 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:48.678106 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:48.678243 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:48.682345 (+ 4102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:59825: BlockingRecv error: recv got EOF from 127.10.242.2:59825 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 06:44:49 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:49 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:49.022193 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:49.013149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:46143)
0810 06:44:49.013618 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:49.013634 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:49.013768 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:49.014096 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:49.014101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:49.014566 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:49.014733 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:49.014743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.016152 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.016158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:49.017094 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:49.017106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.017812 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.017818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:49.017984 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:49.018571 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:49.018590 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:49.021842 (+ 3252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":238,"threads_started":1}
W20260810 06:44:49.022336 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:49.013302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:46143 (local address 127.10.242.62:34183)
0810 06:44:49.013530 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:49.013536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:49.013710 (+ 174us) server_negotiation.cc:407] Connection header received
0810 06:44:49.013880 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:49.013884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:49.013946 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:49.014028 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:49.014879 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.016011 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.017234 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.017661 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.017771 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:49.022054 (+ 4283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:46143: BlockingRecv error: recv got EOF from 127.10.242.3:46143 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 10 06:44:49 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:49 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:49.341272 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:49.332263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:54949)
0810 06:44:49.332875 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:49.332891 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:49.333008 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:49.333297 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:49.333300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:49.333713 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:49.333904 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:49.333918 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.335308 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.335313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:49.336296 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:49.336310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.336946 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.336950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:49.337064 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:49.337609 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:49.337645 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:49.340929 (+ 3284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":269,"threads_started":1}
W20260810 06:44:49.341387 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:49.332391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:54949 (local address 127.10.242.62:34183)
0810 06:44:49.332584 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:49.332588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:49.332974 (+ 386us) server_negotiation.cc:407] Connection header received
0810 06:44:49.333099 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:49.333103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:49.333165 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:49.333268 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:49.334074 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.335152 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.336428 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.336832 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.336924 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:49.341153 (+ 4229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:54949: BlockingRecv error: recv got EOF from 127.10.242.1:54949 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 06:44:49 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:49 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:49.692162 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:49.683441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:59361)
0810 06:44:49.684038 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:49.684053 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:49.684184 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:49.684487 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:49.684492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:49.684969 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:49.685149 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:49.685158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.686237 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.686243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:49.687217 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:49.687230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.687998 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.688004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:49.688167 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:49.688705 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:49.688722 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:49.691822 (+ 3100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":276,"threads_started":1}
W20260810 06:44:49.692301 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:49.683480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:59361 (local address 127.10.242.62:34183)
0810 06:44:49.683648 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:49.683654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:49.684148 (+ 494us) server_negotiation.cc:407] Connection header received
0810 06:44:49.684295 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:49.684300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:49.684366 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:49.684445 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:49.685289 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.686077 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.687360 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:49.687844 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:49.687958 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:49.692058 (+ 4100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:59361: BlockingRecv error: recv got EOF from 127.10.242.2:59361 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:44:50 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:50 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:50.032960 26541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:50.023107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:34447)
0810 06:44:50.023698 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:50.023715 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:50.023831 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:50.024200 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:50.024205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:50.024748 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:50.024980 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:50.024992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.026612 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.026618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:50.027715 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:50.027730 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.028448 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.028454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:50.028620 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:50.029210 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:50.029228 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:50.032663 (+ 3435us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":250,"threads_started":1}
W20260810 06:44:50.033002 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:50.023197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:34447 (local address 127.10.242.62:34183)
0810 06:44:50.023395 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:50.023402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:50.023791 (+ 389us) server_negotiation.cc:407] Connection header received
0810 06:44:50.023955 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:50.023960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:50.024063 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:50.024173 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:50.025240 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.026478 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.027836 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.028302 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.028410 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:50.032808 (+ 4398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:34447: BlockingRecv error: recv got EOF from 127.10.242.3:34447 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 06:44:50 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:50 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:50.352159 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:50.342210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:33547)
0810 06:44:50.342971 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:50.342988 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:50.343142 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:50.343582 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:50.343587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:50.344151 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:50.344339 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:50.344350 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.345732 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.345736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:50.346736 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:50.346750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.347516 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.347523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:50.347695 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:50.348315 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:50.348333 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:50.351813 (+ 3480us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":492,"threads_started":1}
W20260810 06:44:50.352293 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:50.342371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:33547 (local address 127.10.242.62:34183)
0810 06:44:50.342563 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:50.342568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:50.343076 (+ 508us) server_negotiation.cc:407] Connection header received
0810 06:44:50.343322 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:50.343327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:50.343405 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:50.343506 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:50.344489 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.345606 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.346894 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.347334 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.347441 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:50.352021 (+ 4580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:33547: BlockingRecv error: recv got EOF from 127.10.242.1:33547 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:44:50 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:50 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:50.703228 26547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:50.693096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:57529)
0810 06:44:50.693701 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:50.693717 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:50.693832 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:50.694132 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:50.694134 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:50.694566 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:50.694752 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:50.694762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.696460 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.696463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:50.697410 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:50.697422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.698410 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.698417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:50.698569 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:50.699252 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:50.699277 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:50.702906 (+ 3629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":334,"threads_started":1}
W20260810 06:44:50.703591 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:50.693241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:57529 (local address 127.10.242.62:34183)
0810 06:44:50.693451 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:50.693458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:50.693801 (+ 343us) server_negotiation.cc:407] Connection header received
0810 06:44:50.693925 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:50.693930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:50.694014 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:50.694123 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:50.694900 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.696336 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.697551 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:50.698229 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:50.698351 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:50.703144 (+ 4793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:57529: BlockingRecv error: recv got EOF from 127.10.242.2:57529 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:44:51 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:51 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:51.043905 26549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:51.033913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:39437)
0810 06:44:51.034422 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:51.034437 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:51.034567 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:51.034878 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:51.034883 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:51.035334 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:51.035571 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:51.035584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.036840 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.036846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:51.038270 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:51.038287 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.039035 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.039041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:51.039202 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:51.039786 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:51.039807 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:51.043572 (+ 3765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":279,"threads_started":1}
W20260810 06:44:51.043920 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:51.033944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:39437 (local address 127.10.242.62:34183)
0810 06:44:51.034110 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:51.034117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:51.034518 (+ 401us) server_negotiation.cc:407] Connection header received
0810 06:44:51.034677 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:51.034682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:51.034748 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:51.034828 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:51.035711 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.036700 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.038402 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.038851 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.038948 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:51.043738 (+ 4790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:39437: BlockingRecv error: recv got EOF from 127.10.242.3:39437 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 06:44:51 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:51 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:51.363633 26551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:51.353239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:51147)
0810 06:44:51.354084 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:51.354104 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:51.354260 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:51.354677 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:51.354682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:51.355201 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:51.355382 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:51.355393 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.356398 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.356404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:51.357920 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:51.357934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.358705 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.358710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:51.358892 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:51.359464 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:51.359485 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:51.363322 (+ 3837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":576,"threads_started":1}
W20260810 06:44:51.363754 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:51.353270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:51147 (local address 127.10.242.62:34183)
0810 06:44:51.353445 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:51.353452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:51.354184 (+ 732us) server_negotiation.cc:407] Connection header received
0810 06:44:51.354408 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:51.354414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:51.354497 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:51.354607 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:51.355496 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.356267 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.358068 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.358561 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.358656 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:51.363512 (+ 4856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:51147: BlockingRecv error: recv got EOF from 127.10.242.1:51147 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 06:44:51 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:51 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:51.713418 26552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:51.704198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:55597)
0810 06:44:51.704745 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:51.704766 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:51.704906 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:51.705277 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:51.705282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:51.705851 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:51.706043 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:51.706058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.707242 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.707248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:51.708191 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:51.708203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.708891 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.708897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:51.709076 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:51.709627 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:51.709645 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:51.713108 (+ 3463us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":258,"threads_started":1}
W20260810 06:44:51.713587 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:51.704244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:55597 (local address 127.10.242.62:34183)
0810 06:44:51.704449 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:51.704457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:51.704852 (+ 395us) server_negotiation.cc:407] Connection header received
0810 06:44:51.705034 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:51.705040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:51.705124 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:51.705219 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:51.706202 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.707102 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.708328 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:51.708748 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:51.708856 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:51.713312 (+ 4456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:55597: BlockingRecv error: recv got EOF from 127.10.242.2:55597 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 06:44:52 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:52 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:52.055399 26553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:52.045109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:57767)
0810 06:44:52.045810 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:52.045829 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:52.045967 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:52.046330 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:52.046335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:52.046867 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:52.047071 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:52.047085 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.048101 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.048107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:52.049653 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:52.049670 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.050428 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.050435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:52.050625 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:52.051211 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:52.051235 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:52.055141 (+ 3906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":431,"threads_started":1}
W20260810 06:44:52.055614 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:52.045107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:57767 (local address 127.10.242.62:34183)
0810 06:44:52.045388 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:52.045395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:52.045914 (+ 519us) server_negotiation.cc:407] Connection header received
0810 06:44:52.046086 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:52.046090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:52.046167 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:52.046269 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:52.047207 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.047964 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.049783 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.050287 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.050380 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:52.055375 (+ 4995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:57767: BlockingRecv error: recv got EOF from 127.10.242.3:57767 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
Aug 10 06:44:52 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:52 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:52.374285 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:52.364702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:48053)
0810 06:44:52.365323 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:52.365339 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:52.365451 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:52.365813 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:52.365816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:52.366282 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:52.366461 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:52.366472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.367879 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.367882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:52.368848 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:52.368859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.369768 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.369772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:52.369888 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:52.370492 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:52.370510 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:52.373929 (+ 3419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":354,"threads_started":1}
W20260810 06:44:52.374688 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:52.364850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:48053 (local address 127.10.242.62:34183)
0810 06:44:52.365081 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:52.365087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:52.365415 (+ 328us) server_negotiation.cc:407] Connection header received
0810 06:44:52.365567 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:52.365573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:52.365671 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:52.365775 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:52.366640 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.367759 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.368994 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.369634 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.369765 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:52.374125 (+ 4360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:48053: BlockingRecv error: recv got EOF from 127.10.242.1:48053 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 06:44:52 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:52 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:52.723416 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:52.714584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:45103)
0810 06:44:52.715071 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:52.715085 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:52.715216 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:52.715525 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:52.715529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:52.715955 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:52.716120 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:52.716129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.717111 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.717115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:52.718099 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:52.718113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.718860 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.718867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:52.719021 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:52.719617 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:52.719634 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:52.723083 (+ 3449us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":267,"threads_started":1}
W20260810 06:44:52.723537 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:52.714604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:45103 (local address 127.10.242.62:34183)
0810 06:44:52.714765 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:52.714771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:52.715168 (+ 397us) server_negotiation.cc:407] Connection header received
0810 06:44:52.715326 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:52.715332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:52.715398 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:52.715476 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:52.716271 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.716971 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.718266 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:52.718705 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:52.718822 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:52.723291 (+ 4469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:45103: BlockingRecv error: recv got EOF from 127.10.242.2:45103 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 06:44:53 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:53 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:53.071291 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:53.056356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:47789)
0810 06:44:53.056867 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:53.056883 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:53.057011 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:53.057373 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:53.057378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:53.057768 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:53.057930 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:53.057939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.059012 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.059016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:53.059996 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:53.060009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.060732 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.060738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:53.060895 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:53.067474 (+ 6579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:53.067493 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:53.070946 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":296,"threads_started":1}
W20260810 06:44:53.071401 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:53.056475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:47789 (local address 127.10.242.62:34183)
0810 06:44:53.056649 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:53.056655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:53.056957 (+ 302us) server_negotiation.cc:407] Connection header received
0810 06:44:53.057120 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:53.057126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:53.057215 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:53.057297 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:53.058072 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.058870 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.060135 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.060582 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.060698 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:53.071158 (+ 10460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:47789: BlockingRecv error: recv got EOF from 127.10.242.3:47789 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 06:44:53 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:53 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:53.385871 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:53.375307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:35003)
0810 06:44:53.375870 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:53.375885 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:53.375998 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:53.376353 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:53.376357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:53.376803 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:53.376989 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:53.376998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.378363 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.378368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:53.379720 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:53.379739 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.380577 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.380584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:53.380793 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:53.381533 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:53.381559 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:53.385566 (+ 4007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":359,"threads_started":1}
W20260810 06:44:53.386037 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:53.375375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:35003 (local address 127.10.242.62:34183)
0810 06:44:53.375534 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:53.375539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:53.375968 (+ 429us) server_negotiation.cc:407] Connection header received
0810 06:44:53.376100 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:53.376104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:53.376198 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:53.376294 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:53.377137 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.378242 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.379858 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.380414 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.380531 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:53.385798 (+ 5267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:35003: BlockingRecv error: recv got EOF from 127.10.242.1:35003 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 06:44:53 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:53 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:53.733556 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:53.724515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:42959)
0810 06:44:53.725026 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:53.725047 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:53.725206 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:53.725534 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:53.725539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:53.726085 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:53.726337 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:53.726348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.727516 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.727523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:53.728750 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:53.728765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.729424 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.729430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:53.729576 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:53.730110 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:53.730135 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:53.733332 (+ 3197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":282,"threads_started":1}
W20260810 06:44:53.733721 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:53.724631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:42959 (local address 127.10.242.62:34183)
0810 06:44:53.724802 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:53.724807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:53.725128 (+ 321us) server_negotiation.cc:407] Connection header received
0810 06:44:53.725322 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:53.725326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:53.725388 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:53.725484 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:53.726475 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.727346 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.728874 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:53.729286 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:53.729405 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:53.733503 (+ 4098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:42959: BlockingRecv error: recv got EOF from 127.10.242.2:42959 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 06:44:54 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:54 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:54.080874 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:54.072334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:45505)
0810 06:44:54.072813 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:54.072829 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:54.072962 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:54.073302 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:54.073306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:54.073748 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:54.073941 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:54.073950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.075027 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.075032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:54.075940 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:54.075952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.076655 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.076661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:54.076832 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:54.077419 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:54.077436 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:54.080558 (+ 3122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":259,"threads_started":1}
W20260810 06:44:54.080983 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:54.072410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:45505 (local address 127.10.242.62:34183)
0810 06:44:54.072603 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:54.072609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:54.072911 (+ 302us) server_negotiation.cc:407] Connection header received
0810 06:44:54.073077 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:54.073082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:54.073154 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:54.073246 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:54.074089 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.074896 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.076089 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.076509 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.076616 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:54.080762 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:45505: BlockingRecv error: recv got EOF from 127.10.242.3:45505 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 06:44:54 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:54 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:54.396137 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:54.387083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:55545)
0810 06:44:54.387623 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:54.387640 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:54.387756 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:54.388049 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:54.388053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:54.388480 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:54.388814 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:54.388831 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.390142 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.390146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:54.391121 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:54.391135 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.391884 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.391889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:54.392030 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:54.392625 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:54.392644 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:54.395890 (+ 3246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":278,"threads_started":1}
W20260810 06:44:54.396303 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:54.387180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:55545 (local address 127.10.242.62:34183)
0810 06:44:54.387370 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:54.387375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:54.387731 (+ 356us) server_negotiation.cc:407] Connection header received
0810 06:44:54.387858 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:54.387862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:54.387940 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:54.388020 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:54.388963 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.390001 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.391247 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.391722 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.391819 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:54.396084 (+ 4265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:55545: BlockingRecv error: recv got EOF from 127.10.242.1:55545 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 06:44:54 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:54 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:54.743585 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:54.734499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:37041)
0810 06:44:54.735038 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:54.735054 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:54.735192 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:54.735498 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:54.735502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:54.735946 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:54.736119 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:54.736146 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.737242 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.737247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:54.738512 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:54.738526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.739285 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.739291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:54.739468 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:54.740017 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:54.740034 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:54.743258 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":273,"threads_started":1}
W20260810 06:44:54.743739 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:54.734570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:37041 (local address 127.10.242.62:34183)
0810 06:44:54.734766 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:54.734773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:54.735134 (+ 361us) server_negotiation.cc:407] Connection header received
0810 06:44:54.735304 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:54.735308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:54.735376 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:54.735454 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:54.736279 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.737101 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.738653 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:54.739115 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:54.739226 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:54.743475 (+ 4249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:37041: BlockingRecv error: recv got EOF from 127.10.242.2:37041 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 06:44:55 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:55 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:55.090843 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:55.081806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:59339)
0810 06:44:55.082367 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:55.082383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:55.082513 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:55.082836 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:55.082841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:55.083294 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:55.083466 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:55.083475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.084508 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.084514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:55.085883 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:55.085899 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.086644 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.086651 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:55.086861 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:55.087458 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:55.087476 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:55.090591 (+ 3115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":322,"threads_started":1}
W20260810 06:44:55.091027 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:55.081909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:59339 (local address 127.10.242.62:34183)
0810 06:44:55.082099 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:55.082105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:55.082465 (+ 360us) server_negotiation.cc:407] Connection header received
0810 06:44:55.082623 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:55.082628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:55.082702 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:55.082785 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:55.083602 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.084381 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.086000 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.086466 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.086599 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:55.090808 (+ 4209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:59339: BlockingRecv error: recv got EOF from 127.10.242.3:59339 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 06:44:55 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:55 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:55.405365 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:55.397051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:38681)
0810 06:44:55.397573 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:55.397593 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:55.397761 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:55.398187 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:55.398193 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:55.398607 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:55.398781 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:55.398791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.399717 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.399720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:55.400602 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:55.400612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.401239 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.401243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:55.401364 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:55.401873 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:55.401889 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:55.405136 (+ 3247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":276,"threads_started":1}
W20260810 06:44:55.405555 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:55.397105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:38681 (local address 127.10.242.62:34183)
0810 06:44:55.397304 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:55.397311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:55.397685 (+ 374us) server_negotiation.cc:407] Connection header received
0810 06:44:55.397904 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:55.397909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:55.398013 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:55.398106 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:55.398907 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.399612 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.400724 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.401131 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.401229 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:55.405314 (+ 4085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:38681: BlockingRecv error: recv got EOF from 127.10.242.1:38681 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 06:44:55 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:55 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:55.752941 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:55.744562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:41409)
0810 06:44:55.745107 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:55.745126 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:55.745276 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:55.745557 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:55.745560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:55.745958 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:55.746115 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:55.746124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.747162 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.747165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:55.748129 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:55.748143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.748851 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.748855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:55.748980 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:55.749520 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:55.749536 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:55.752721 (+ 3185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":296,"threads_started":1}
W20260810 06:44:55.753109 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:55.744691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:41409 (local address 127.10.242.62:34183)
0810 06:44:55.744875 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:55.744881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:55.745200 (+ 319us) server_negotiation.cc:407] Connection header received
0810 06:44:55.745378 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:55.745381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:55.745445 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:55.745532 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:55.746263 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.747048 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.748259 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:55.748730 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:55.748821 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:55.752913 (+ 4092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:41409: BlockingRecv error: recv got EOF from 127.10.242.2:41409 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 06:44:56 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:56 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.3@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:56.100173 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:56.091693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.3:56881)
0810 06:44:56.092171 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:56.092188 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:56.092305 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:56.092558 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:56.092561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:56.092997 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:56.093150 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:56.093159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.094208 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.094213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:56.095367 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:56.095384 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.096009 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.096012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:56.096134 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:56.096671 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:56.096689 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:56.099950 (+ 3261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":257,"threads_started":1}
W20260810 06:44:56.100435 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:56.091774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:56881 (local address 127.10.242.62:34183)
0810 06:44:56.091937 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:56.091942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:56.092250 (+ 308us) server_negotiation.cc:407] Connection header received
0810 06:44:56.092388 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:56.092391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:56.092451 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:56.092530 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:56.093271 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.094042 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.095490 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.095898 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.095987 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:56.100198 (+ 4211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.3:56881: BlockingRecv error: recv got EOF from 127.10.242.3:56881 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 06:44:56 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:56 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.1@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:56.415359 26572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:56.406451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.1:33011)
0810 06:44:56.406914 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:56.406930 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:56.407042 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:56.407398 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:56.407403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:56.407934 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:56.408128 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:56.408139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.409163 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.409167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:56.410085 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:56.410098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.410890 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.410896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:56.411072 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:56.411576 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:56.411593 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:56.415016 (+ 3423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":219,"threads_started":1}
W20260810 06:44:56.415467 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:56.406511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:33011 (local address 127.10.242.62:34183)
0810 06:44:56.406841 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:56.406847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:56.407012 (+ 165us) server_negotiation.cc:407] Connection header received
0810 06:44:56.407152 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:56.407156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:56.407248 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:56.407337 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:56.408278 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.409012 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.410253 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.410752 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.410862 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:56.415232 (+ 4370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.1:33011: BlockingRecv error: recv got EOF from 127.10.242.1:33011 (error 108)
Metrics: {"server-negotiator.queue_time_us":207}
Aug 10 06:44:56 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:44:56 dist-test-slave-zj8q krb5kdc[24747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.242.2@KRBTEST.COM for oryx/127.10.242.62@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:44:56.762883 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:56.754095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:34183 (local address 127.10.242.2:43905)
0810 06:44:56.754641 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:56.754658 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:56.754784 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:56.755134 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:56.755139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:56.755641 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:56.755927 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:56.755943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.757128 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.757132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:56.758062 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:56.758075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.758724 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.758728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:56.758861 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:56.759414 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:44:56.759432 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:56.762624 (+ 3192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.242.62:34183: Server oryx/127.10.242.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":260,"threads_started":1}
W20260810 06:44:56.763024 26491 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:56.754237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:43905 (local address 127.10.242.62:34183)
0810 06:44:56.754388 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:56.754393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:56.754725 (+ 332us) server_negotiation.cc:407] Connection header received
0810 06:44:56.754884 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:56.754887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:56.754967 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:56.755050 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:56.756079 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.756992 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.758194 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.758592 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.758696 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:56.762827 (+ 4131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.242.2:43905: BlockingRecv error: recv got EOF from 127.10.242.2:43905 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:44:56.905493 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 25738
I20260810 06:44:56.917200 26577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:56.906422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40984 (local address 127.10.242.1:42353)
0810 06:44:56.907029 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:56.907037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:56.907067 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:44:56.907128 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:56.907135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:56.907360 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:56.907522 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:56.908875 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.909951 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.911123 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.911707 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.911805 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:56.914252 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:56.914262 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:56.914276 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:56.914313 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:56.916718 (+ 2405us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 06:44:56.916985 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40984: Request ticket server kudu/127.10.242.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":258,"threads_started":1}
W20260810 06:44:56.917299 26577 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40984: Request ticket server kudu/127.10.242.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 06:44:56.917330 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 25738: Not authorized: Client connection negotiation failed: client connection to 127.10.242.1:42353: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 06:44:56.917519 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 25738
I20260810 06:44:56.944416 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 26012
I20260810 06:44:56.957089 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:56.945618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.10.242.2:34913)
0810 06:44:56.946313 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:56.946321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:56.946352 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:44:56.946417 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:56.946424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:56.946653 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:56.946829 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:56.948480 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.949614 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.950923 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.951495 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.951596 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:56.953845 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:56.953854 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:56.953869 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:56.953909 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:56.956571 (+ 2662us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 06:44:56.956831 (+ 260us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44844: Request ticket server kudu/127.10.242.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":316,"threads_started":1}
W20260810 06:44:56.957201 26578 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44844: Request ticket server kudu/127.10.242.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 06:44:56.957204 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 26012: Not authorized: Client connection negotiation failed: client connection to 127.10.242.2:34913: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 06:44:56.957358 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 26012
I20260810 06:44:56.980697 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 26240
I20260810 06:44:56.992355 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:56.981700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45646 (local address 127.10.242.3:40917)
0810 06:44:56.982274 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:56.982281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:56.982310 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:44:56.982377 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:56.982383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:56.982622 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:56.982802 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:56.984119 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.985122 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.986330 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:56.986888 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:56.987007 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:56.989326 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:56.989335 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:56.989349 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:44:56.989406 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:56.991715 (+ 2309us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 06:44:56.992027 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45646: Request ticket server kudu/127.10.242.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":226,"threads_started":1}
W20260810 06:44:56.992378 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 26240: Not authorized: Client connection negotiation failed: client connection to 127.10.242.3:40917: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.242.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 06:44:56.992447 26581 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45646: Request ticket server kudu/127.10.242.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 06:44:56.992527 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 26240
I20260810 06:44:57.014171 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 25624
I20260810 06:44:57.015499 25729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:57.133416 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 25624
2026-08-10T06:44:57Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21214 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T06:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:57Z Disabled control of system clock
I20260810 06:44:57.228607 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:46657
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:40401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:46657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:44:57.552757 26592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:57.553050 26592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:57.553153 26592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:57.560241 26592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:57.560345 26592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:57.560389 26592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:57.560444 26592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:57.560489 26592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:57.569454 26592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:46657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:46657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:57.570770 26592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:57.572515 26592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:57.583858 26599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:57.583945 26600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:57.583922 26602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:57.584450 26592 server_base.cc:1034] running on GCE node
I20260810 06:44:57.585230 26592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:57.586649 26592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:57.587848 26592 hybrid_clock.cc:648] HybridClock initialized: now 1786344297587806 us; error 36 us; skew 500 ppm
I20260810 06:44:57.588194 26592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:57.590353 26592 webserver.cc:460] Webserver started at http://127.10.242.62:39929/ using document root <none> and password file <none>
I20260810 06:44:57.591022 26592 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:57.591149 26592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:57.591463 26592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:57.594599 26592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "dcc7545963d5443b831af3d2762ddd26"
format_stamp: "Formatted at 2026-08-10 06:44:57 on dist-test-slave-zj8q"
server_key: "3e5a1bb3c95cba2158f9faa47cd01c14"
server_key_iv: "d1c89b05183f843d0ba91f9dc390d6c9"
server_key_version: "encryptionkey@0"
I20260810 06:44:57.595360 26592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "dcc7545963d5443b831af3d2762ddd26"
format_stamp: "Formatted at 2026-08-10 06:44:57 on dist-test-slave-zj8q"
server_key: "3e5a1bb3c95cba2158f9faa47cd01c14"
server_key_iv: "d1c89b05183f843d0ba91f9dc390d6c9"
server_key_version: "encryptionkey@0"
I20260810 06:44:57.600288 26592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:44:57.603540 26611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:57.604764 26592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:44:57.604923 26592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "dcc7545963d5443b831af3d2762ddd26"
format_stamp: "Formatted at 2026-08-10 06:44:57 on dist-test-slave-zj8q"
server_key: "3e5a1bb3c95cba2158f9faa47cd01c14"
server_key_iv: "d1c89b05183f843d0ba91f9dc390d6c9"
server_key_version: "encryptionkey@0"
I20260810 06:44:57.605134 26592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:57.618436 26592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:57.619169 26592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:57.619419 26592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:57.636682 26592 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:46657
I20260810 06:44:57.636703 26698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:46657 every 8 connection(s)
I20260810 06:44:57.637904 26592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:44:57.640520 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 26592
I20260810 06:44:57.640722 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:44:57.641542 11208 external_mini_cluster.cc:1428] Setting key 14703199e376900b72d3d08e56fa363e
I20260810 06:44:57.642009 26700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:44:57.654347 26700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26: Bootstrap starting.
I20260810 06:44:57.655115 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:57.643662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51156 (local address 127.10.242.62:46657)
0810 06:44:57.644640 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:57.644651 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:57.644695 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:44:57.646606 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:57.646629 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:57.647087 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:57.647441 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:57.648034 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:57.649346 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:57.650751 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:57.651449 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:57.651654 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:57.652836 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:57.652843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:57.652857 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:57.653359 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:57.653435 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:57.653456 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:57.653571 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:57.653780 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 06:44:57.654069 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":341,"threads_started":1}
I20260810 06:44:57.657366 26700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26: Neither blocks nor log segments found. Creating new log.
I20260810 06:44:57.658463 26700 log.cc:824] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26: Log is configured to *not* fsync() on all Append() calls
I20260810 06:44:57.660992 26700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26: No bootstrap required, opened a new log
I20260810 06:44:57.668143 26700 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc7545963d5443b831af3d2762ddd26" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46657 } }
I20260810 06:44:57.668524 26700 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:44:57.668664 26700 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc7545963d5443b831af3d2762ddd26, State: Initialized, Role: FOLLOWER
I20260810 06:44:57.669212 26700 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc7545963d5443b831af3d2762ddd26" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46657 } }
I20260810 06:44:57.669464 26700 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:44:57.669590 26700 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:44:57.669765 26700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:44:57.671813 26700 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc7545963d5443b831af3d2762ddd26" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46657 } }
I20260810 06:44:57.672258 26700 leader_election.cc:302] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcc7545963d5443b831af3d2762ddd26; no voters:
I20260810 06:44:57.672775 26700 leader_election.cc:288] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:44:57.672904 26706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:44:57.673556 26706 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [term 1 LEADER]: Becoming Leader. State: Replica: dcc7545963d5443b831af3d2762ddd26, State: Running, Role: LEADER
I20260810 06:44:57.674075 26706 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc7545963d5443b831af3d2762ddd26" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46657 } }
I20260810 06:44:57.674862 26700 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:44:57.676888 26709 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcc7545963d5443b831af3d2762ddd26. Latest consensus state: current_term: 1 leader_uuid: "dcc7545963d5443b831af3d2762ddd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc7545963d5443b831af3d2762ddd26" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46657 } } }
I20260810 06:44:57.677300 26709 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:57.677170 26708 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcc7545963d5443b831af3d2762ddd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcc7545963d5443b831af3d2762ddd26" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46657 } } }
I20260810 06:44:57.677484 26708 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26 [sys.catalog]: This master's current role is: LEADER
I20260810 06:44:57.678792 26719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:44:57.683941 26719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:44:57.692879 26719 catalog_manager.cc:1347] Generated new cluster ID: e615abdf25384635b130bc8b22a28cda
I20260810 06:44:57.692986 26719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:44:57.706270 26719 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:44:57.707436 26719 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:44:57.713640 26719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcc7545963d5443b831af3d2762ddd26: Generated new TSK 0
I20260810 06:44:57.714372 26719 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:44:57.726359 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46657
--builtin_ntp_servers=127.10.242.20:40401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 06:44:58.053161 26742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:58.053413 26742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:58.053534 26742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:58.060516 26742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:58.060604 26742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:58.060740 26742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:44:58.069334 26742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:58.070520 26742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:58.072306 26742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:58.084760 26749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:58.084791 26755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:58.085002 26748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:58.085675 26742 server_base.cc:1034] running on GCE node
I20260810 06:44:58.086324 26742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:58.087287 26742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:58.088479 26742 hybrid_clock.cc:648] HybridClock initialized: now 1786344298088429 us; error 43 us; skew 500 ppm
I20260810 06:44:58.088789 26742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:58.093878 26742 webserver.cc:460] Webserver started at http://127.10.242.1:32955/ using document root <none> and password file <none>
I20260810 06:44:58.094609 26742 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:58.094717 26742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:58.095039 26742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:58.097841 26742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "2ee5375e4d554173bd29ad65495a7599"
format_stamp: "Formatted at 2026-08-10 06:44:58 on dist-test-slave-zj8q"
server_key: "35c41f2a04372beb493f6f450201093b"
server_key_iv: "c4b9a8244bf7422417d1a2141c18ceef"
server_key_version: "encryptionkey@0"
I20260810 06:44:58.098661 26742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "2ee5375e4d554173bd29ad65495a7599"
format_stamp: "Formatted at 2026-08-10 06:44:58 on dist-test-slave-zj8q"
server_key: "35c41f2a04372beb493f6f450201093b"
server_key_iv: "c4b9a8244bf7422417d1a2141c18ceef"
server_key_version: "encryptionkey@0"
I20260810 06:44:58.103624 26742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:44:58.106993 26767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:58.108314 26742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:44:58.108446 26742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "2ee5375e4d554173bd29ad65495a7599"
format_stamp: "Formatted at 2026-08-10 06:44:58 on dist-test-slave-zj8q"
server_key: "35c41f2a04372beb493f6f450201093b"
server_key_iv: "c4b9a8244bf7422417d1a2141c18ceef"
server_key_version: "encryptionkey@0"
I20260810 06:44:58.108623 26742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:58.132326 26742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:58.133141 26742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:58.133409 26742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:58.134377 26742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:58.135811 26742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:58.135898 26742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:58.135982 26742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:58.136034 26742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:58.166092 26742 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:36131
I20260810 06:44:58.166184 26954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:36131 every 8 connection(s)
I20260810 06:44:58.167423 26742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:44:58.172286 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 26742
I20260810 06:44:58.172451 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:44:58.173062 11208 external_mini_cluster.cc:1428] Setting key 1fee35002e1d01c16315456f282b2311
I20260810 06:44:58.176174 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46657
--builtin_ntp_servers=127.10.242.20:40401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 06:44:58.180480 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:58.169819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:47499 (local address 127.10.242.62:46657)
0810 06:44:58.170259 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:58.170265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:58.171372 (+ 1107us) server_negotiation.cc:407] Connection header received
0810 06:44:58.173233 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:58.173240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:58.173319 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:58.173415 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:58.174740 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:58.175861 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:58.177473 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:58.178092 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:58.178257 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:58.179694 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:58.179700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:58.179704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:58.179724 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:58.179770 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:58.179785 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:58.179862 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:58.180049 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 06:44:58.180172 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":203,"threads_started":1}
I20260810 06:44:58.181347 26968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:58.170347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:46657 (local address 127.10.242.1:47499)
0810 06:44:58.171222 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:58.171258 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:58.172953 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:58.173652 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:58.173660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:58.173760 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:58.174546 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:58.174573 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:58.176029 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:58.176034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:58.177320 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:58.177334 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:58.178329 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:58.178334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:58.178618 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:58.179346 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:58.179350 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:58.179401 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:58.179499 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:58.179872 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:58.179877 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:58.179882 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:44:58.180007 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 06:44:58.180302 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":270,"threads_started":1}
I20260810 06:44:58.182787 26960 heartbeater.cc:347] Connected to a master server at 127.10.242.62:46657
I20260810 06:44:58.183064 26960 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:58.183792 26960 heartbeater.cc:511] Master 127.10.242.62:46657 requested a full tablet report, sending...
I20260810 06:44:58.185695 26644 ts_manager.cc:194] Registered new tserver with Master: 2ee5375e4d554173bd29ad65495a7599 (127.10.242.1:36131)
I20260810 06:44:58.187286 26644 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.1:47499
W20260810 06:44:58.501660 26969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:58.501905 26969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:58.502024 26969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:58.509009 26969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:58.509120 26969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:58.509261 26969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:44:58.517961 26969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:58.519162 26969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:58.520936 26969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:58.533258 26978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:58.533324 26976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:58.533344 26981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:58.534284 26969 server_base.cc:1034] running on GCE node
I20260810 06:44:58.535025 26969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:58.535934 26969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:58.537128 26969 hybrid_clock.cc:648] HybridClock initialized: now 1786344298537072 us; error 46 us; skew 500 ppm
I20260810 06:44:58.537431 26969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:58.539984 26969 webserver.cc:460] Webserver started at http://127.10.242.2:33081/ using document root <none> and password file <none>
I20260810 06:44:58.540891 26969 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:58.541028 26969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:58.541420 26969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:58.545423 26969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "33f85d270edf410dbd966a038780f86b"
format_stamp: "Formatted at 2026-08-10 06:44:58 on dist-test-slave-zj8q"
server_key: "fb88e7492b0a3ace01a1493e73a18f2e"
server_key_iv: "2657e4e3efc50de37c1ab786b806a4a3"
server_key_version: "encryptionkey@0"
I20260810 06:44:58.546726 26969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "33f85d270edf410dbd966a038780f86b"
format_stamp: "Formatted at 2026-08-10 06:44:58 on dist-test-slave-zj8q"
server_key: "fb88e7492b0a3ace01a1493e73a18f2e"
server_key_iv: "2657e4e3efc50de37c1ab786b806a4a3"
server_key_version: "encryptionkey@0"
I20260810 06:44:58.552536 26969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:44:58.557335 26989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:58.560945 26969 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260810 06:44:58.561113 26969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "33f85d270edf410dbd966a038780f86b"
format_stamp: "Formatted at 2026-08-10 06:44:58 on dist-test-slave-zj8q"
server_key: "fb88e7492b0a3ace01a1493e73a18f2e"
server_key_iv: "2657e4e3efc50de37c1ab786b806a4a3"
server_key_version: "encryptionkey@0"
I20260810 06:44:58.561331 26969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:58.589710 26969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:58.590824 26969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:58.591163 26969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:58.592281 26969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:58.593815 26969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:58.593906 26969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:58.594012 26969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:58.594081 26969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:58.623953 26969 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:35399
I20260810 06:44:58.624022 27179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:35399 every 8 connection(s)
I20260810 06:44:58.625232 26969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:44:58.633134 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 26969
I20260810 06:44:58.633294 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:44:58.633725 11208 external_mini_cluster.cc:1428] Setting key d1a2cd63012010e42b8b6314598ba504
I20260810 06:44:58.636600 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46657
--builtin_ntp_servers=127.10.242.20:40401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 06:44:58.637485 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:58.627670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:52627 (local address 127.10.242.62:46657)
0810 06:44:58.627838 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:58.627843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:58.629188 (+ 1345us) server_negotiation.cc:407] Connection header received
0810 06:44:58.630990 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:58.630994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:58.631058 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:58.631140 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:58.632106 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:58.632848 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:58.634507 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:58.635081 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:58.635213 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:58.636580 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:58.636587 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:58.636592 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:58.636615 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:58.636656 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:58.636716 (+ 60us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:58.636810 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:58.637090 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 06:44:58.637189 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:44:58.638655 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:58.628144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:46657 (local address 127.10.242.2:52627)
0810 06:44:58.629028 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:58.629073 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:58.630742 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:58.631326 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:58.631334 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:58.631416 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:58.631979 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:58.631995 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:58.632972 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:58.632976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:58.634346 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:58.634362 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:58.635285 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:58.635291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:58.635557 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:58.636235 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:58.636239 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:58.636289 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:58.636387 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:58.636869 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:58.636881 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:58.636888 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:44:58.637054 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 06:44:58.637413 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":259,"threads_started":1}
I20260810 06:44:58.643412 27180 heartbeater.cc:347] Connected to a master server at 127.10.242.62:46657
I20260810 06:44:58.643723 27180 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:58.644565 27180 heartbeater.cc:511] Master 127.10.242.62:46657 requested a full tablet report, sending...
I20260810 06:44:58.646143 26644 ts_manager.cc:194] Registered new tserver with Master: 33f85d270edf410dbd966a038780f86b (127.10.242.2:35399)
I20260810 06:44:58.647145 26644 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.2:52627
W20260810 06:44:58.961899 27186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:58.962178 27186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:58.962296 27186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:58.969195 27186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:58.969295 27186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:58.969419 27186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:44:58.978111 27186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:58.979339 27186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:58.981186 27186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:58.993665 27194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:58.993696 27195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:58.993628 27197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:58.994338 27186 server_base.cc:1034] running on GCE node
I20260810 06:44:58.994899 27186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:58.995738 27186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:58.996937 27186 hybrid_clock.cc:648] HybridClock initialized: now 1786344298996894 us; error 28 us; skew 500 ppm
I20260810 06:44:58.997232 27186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:58.999264 27186 webserver.cc:460] Webserver started at http://127.10.242.3:43253/ using document root <none> and password file <none>
I20260810 06:44:58.999907 27186 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:59.000010 27186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:59.000327 27186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:59.003079 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "62949289d2ce429785d5d6e9d30a9db5"
format_stamp: "Formatted at 2026-08-10 06:44:58 on dist-test-slave-zj8q"
server_key: "1cf9883b765bafbc9ea805b9dc439204"
server_key_iv: "4386a4feae5899310a0beba99e3b8ac4"
server_key_version: "encryptionkey@0"
I20260810 06:44:59.003842 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "62949289d2ce429785d5d6e9d30a9db5"
format_stamp: "Formatted at 2026-08-10 06:44:58 on dist-test-slave-zj8q"
server_key: "1cf9883b765bafbc9ea805b9dc439204"
server_key_iv: "4386a4feae5899310a0beba99e3b8ac4"
server_key_version: "encryptionkey@0"
I20260810 06:44:59.008808 27186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:44:59.012115 27206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:59.013458 27186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:44:59.013597 27186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "62949289d2ce429785d5d6e9d30a9db5"
format_stamp: "Formatted at 2026-08-10 06:44:58 on dist-test-slave-zj8q"
server_key: "1cf9883b765bafbc9ea805b9dc439204"
server_key_iv: "4386a4feae5899310a0beba99e3b8ac4"
server_key_version: "encryptionkey@0"
I20260810 06:44:59.013794 27186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:44:59.033030 27186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:44:59.033823 27186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:44:59.034096 27186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:44:59.035022 27186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:44:59.036471 27186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:44:59.036569 27186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:59.036682 27186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:44:59.036753 27186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:59.066462 27186 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:37693
I20260810 06:44:59.066484 27400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:37693 every 8 connection(s)
I20260810 06:44:59.067806 27186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:44:59.073724 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 27186
I20260810 06:44:59.073894 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:44:59.074371 11208 external_mini_cluster.cc:1428] Setting key 36d3a2115c718596b4822f93f669b82e
I20260810 06:44:59.079661 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:59.070303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:57847 (local address 127.10.242.62:46657)
0810 06:44:59.070485 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:59.070489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:59.071815 (+ 1326us) server_negotiation.cc:407] Connection header received
0810 06:44:59.073484 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:59.073491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:59.073580 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:59.073699 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:59.074787 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.075662 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.077403 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.077801 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.077894 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:59.079045 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:59.079050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:59.079053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:59.079070 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:59.079102 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:59.079114 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:59.079191 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:59.079336 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 06:44:59.079414 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:44:59.080688 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:59.070779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:46657 (local address 127.10.242.3:57847)
0810 06:44:59.071662 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0810 06:44:59.071703 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:44:59.073228 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:44:59.073865 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:44:59.073876 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:44:59.073972 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:44:59.074611 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:59.074630 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.075815 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.075819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:59.077278 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:44:59.077290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.077965 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.077970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:44:59.078200 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:59.078766 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:44:59.078769 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:44:59.078804 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:44:59.078878 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:44:59.079181 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:44:59.079187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:44:59.079192 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:44:59.079310 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 06:44:59.079576 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":299,"threads_started":1}
I20260810 06:44:59.081893 27402 heartbeater.cc:347] Connected to a master server at 127.10.242.62:46657
I20260810 06:44:59.082144 27402 heartbeater.cc:464] Registering TS with master...
I20260810 06:44:59.082908 27402 heartbeater.cc:511] Master 127.10.242.62:46657 requested a full tablet report, sending...
I20260810 06:44:59.084164 26644 ts_manager.cc:194] Registered new tserver with Master: 62949289d2ce429785d5d6e9d30a9db5 (127.10.242.3:37693)
I20260810 06:44:59.084911 26644 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.3:57847
I20260810 06:44:59.089540 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:44:59.095371 26966 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:44:59.093832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51160 (local address 127.10.242.62:46657)
0810 06:44:59.094000 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:59.094005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:59.094485 (+ 480us) server_negotiation.cc:407] Connection header received
0810 06:44:59.094643 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:59.094647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:59.094709 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:59.094785 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:59.095137 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51160: BlockingRecv error: recv got EOF from 127.0.0.1:51160 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:44:59.096470 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 26742
I20260810 06:44:59.103410 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:59.097300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38952 (local address 127.10.242.1:36131)
0810 06:44:59.097729 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:59.097736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:59.097759 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:44:59.097812 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:59.097818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:59.098005 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:59.098131 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:59.098463 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.099424 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.100567 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.101094 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.101184 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:59.102032 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:59.102039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:59.102051 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:59.102678 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:59.102725 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:59.102745 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:59.102826 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:59.102938 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 06:44:59.103091 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":210,"threads_started":1}
I20260810 06:44:59.104537 26948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:59.185662 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 26742
I20260810 06:44:59.211659 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 26969
I20260810 06:44:59.219270 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:59.212517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37762 (local address 127.10.242.2:35399)
0810 06:44:59.213011 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:59.213019 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:59.213047 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:44:59.213116 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:59.213122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:59.213349 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:59.213483 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:59.213878 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.214950 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.216206 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.216757 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.216862 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:59.217835 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:59.217845 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:59.217860 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:59.218541 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:59.218595 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:59.218615 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:59.218696 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:59.218819 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 06:44:59.218946 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":241,"threads_started":1}
I20260810 06:44:59.220492 27172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:59.300057 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 26969
I20260810 06:44:59.323657 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 27186
I20260810 06:44:59.331866 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:44:59.324703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50032 (local address 127.10.242.3:37693)
0810 06:44:59.325365 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0810 06:44:59.325371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:44:59.325404 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:44:59.325465 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:44:59.325471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:44:59.325741 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:44:59.325864 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:44:59.326342 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.327394 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.328666 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:44:59.329243 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:44:59.329339 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:44:59.330499 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:44:59.330508 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:44:59.330530 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:44:59.331103 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:44:59.331155 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:44:59.331177 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:44:59.331270 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:44:59.331371 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 06:44:59.331480 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":354,"threads_started":1}
I20260810 06:44:59.333139 27391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:59.414875 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 27186
I20260810 06:44:59.436396 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 26592
I20260810 06:44:59.437698 26693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:44:59.538902 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 26592
2026-08-10T06:44:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2395 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T06:44:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:44:59Z Disabled control of system clock
I20260810 06:44:59.623709 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:35159
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:35159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:44:59.943545 27437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:44:59.943830 27437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:44:59.943954 27437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:44:59.950979 27437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:44:59.951079 27437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:44:59.951141 27437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:44:59.951224 27437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:44:59.951280 27437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:44:59.960184 27437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:35159
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:35159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:44:59.961432 27437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:44:59.963306 27437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:44:59.974452 27443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:59.974483 27444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:44:59.974538 27446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:44:59.975446 27437 server_base.cc:1034] running on GCE node
I20260810 06:44:59.976136 27437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:44:59.977348 27437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:44:59.978536 27437 hybrid_clock.cc:648] HybridClock initialized: now 1786344299978492 us; error 37 us; skew 500 ppm
I20260810 06:44:59.978857 27437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:44:59.980923 27437 webserver.cc:460] Webserver started at http://127.10.242.62:45111/ using document root <none> and password file <none>
I20260810 06:44:59.981575 27437 fs_manager.cc:359] Metadata directory not provided
I20260810 06:44:59.981675 27437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:44:59.981988 27437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:44:59.984776 27437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "770d81948d4a423ba1ceb720d9ad82c0"
format_stamp: "Formatted at 2026-08-10 06:44:59 on dist-test-slave-zj8q"
server_key: "69ea0726ee56cd8df86fff0142a4c9e3"
server_key_iv: "a7cc28f82a3114d4efe75c1cfe2272b5"
server_key_version: "encryptionkey@0"
I20260810 06:44:59.985551 27437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "770d81948d4a423ba1ceb720d9ad82c0"
format_stamp: "Formatted at 2026-08-10 06:44:59 on dist-test-slave-zj8q"
server_key: "69ea0726ee56cd8df86fff0142a4c9e3"
server_key_iv: "a7cc28f82a3114d4efe75c1cfe2272b5"
server_key_version: "encryptionkey@0"
I20260810 06:44:59.990314 27437 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 06:44:59.993628 27454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:44:59.994859 27437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:44:59.995003 27437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "770d81948d4a423ba1ceb720d9ad82c0"
format_stamp: "Formatted at 2026-08-10 06:44:59 on dist-test-slave-zj8q"
server_key: "69ea0726ee56cd8df86fff0142a4c9e3"
server_key_iv: "a7cc28f82a3114d4efe75c1cfe2272b5"
server_key_version: "encryptionkey@0"
I20260810 06:44:59.995208 27437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:00.000501 27437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:00.001214 27437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:00.001482 27437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:00.025112 27437 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:35159
I20260810 06:45:00.025137 27531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:35159 every 8 connection(s)
I20260810 06:45:00.026784 27437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:00.031122 27532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:00.038609 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 27437
I20260810 06:45:00.038838 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:45:00.039461 11208 external_mini_cluster.cc:1428] Setting key 43c02d0cc47ce7a7d245d52b688ee3c9
I20260810 06:45:00.045971 27532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0: Bootstrap starting.
I20260810 06:45:00.048381 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:00.041488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49050 (local address 127.10.242.62:35159)
0810 06:45:00.042654 (+ 1166us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:00.042667 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:00.042726 (+ 59us) server_negotiation.cc:407] Connection header received
0810 06:45:00.044458 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:00.044482 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:00.044943 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:00.045256 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:00.046368 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:00.046377 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:00.046395 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:00.046426 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:00.046507 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:00.046531 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:00.046764 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:00.046887 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 06:45:00.047184 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":362,"threads_started":1}
I20260810 06:45:00.050122 27532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:00.051469 27532 log.cc:824] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:00.054513 27532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0: No bootstrap required, opened a new log
I20260810 06:45:00.062350 27532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d81948d4a423ba1ceb720d9ad82c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 35159 } }
I20260810 06:45:00.062683 27532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:00.062760 27532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770d81948d4a423ba1ceb720d9ad82c0, State: Initialized, Role: FOLLOWER
I20260810 06:45:00.063246 27532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d81948d4a423ba1ceb720d9ad82c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 35159 } }
I20260810 06:45:00.063437 27532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:00.063529 27532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:00.063643 27532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:00.065658 27532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d81948d4a423ba1ceb720d9ad82c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 35159 } }
I20260810 06:45:00.066078 27532 leader_election.cc:302] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 770d81948d4a423ba1ceb720d9ad82c0; no voters:
I20260810 06:45:00.066725 27532 leader_election.cc:288] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:00.066884 27542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:00.067701 27542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [term 1 LEADER]: Becoming Leader. State: Replica: 770d81948d4a423ba1ceb720d9ad82c0, State: Running, Role: LEADER
I20260810 06:45:00.068205 27542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d81948d4a423ba1ceb720d9ad82c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 35159 } }
I20260810 06:45:00.068743 27532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:00.070896 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 770d81948d4a423ba1ceb720d9ad82c0. Latest consensus state: current_term: 1 leader_uuid: "770d81948d4a423ba1ceb720d9ad82c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d81948d4a423ba1ceb720d9ad82c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 35159 } } }
I20260810 06:45:00.071271 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:00.071527 27543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "770d81948d4a423ba1ceb720d9ad82c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "770d81948d4a423ba1ceb720d9ad82c0" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 35159 } } }
I20260810 06:45:00.071945 27543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:00.073593 27562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:00.078603 27562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:00.086632 27562 catalog_manager.cc:1347] Generated new cluster ID: af6e31d03eb446f093f91d1919f61fcb
I20260810 06:45:00.086735 27562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:00.100090 27562 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:00.100533 27562 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:00.109648 27562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 770d81948d4a423ba1ceb720d9ad82c0: Generated new TSK 0
I20260810 06:45:00.110649 27562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:45:00.120568 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:35159
--builtin_ntp_servers=127.10.242.20:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:45:00.455693 27584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:00.455976 27584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:00.456106 27584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:00.463173 27584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:00.463276 27584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:00.463425 27584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:45:00.472455 27584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:35159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:00.473801 27584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:00.475562 27584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:00.488062 27597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:00.488122 27601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:00.488487 27596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:00.488823 27584 server_base.cc:1034] running on GCE node
I20260810 06:45:00.489332 27584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:00.490141 27584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:00.491364 27584 hybrid_clock.cc:648] HybridClock initialized: now 1786344300491308 us; error 45 us; skew 500 ppm
I20260810 06:45:00.491657 27584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:45:00.493880 27584 webserver.cc:460] Webserver started at http://127.10.242.1:37077/ using document root <none> and password file <none>
I20260810 06:45:00.494570 27584 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:00.494675 27584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:00.494984 27584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:00.499897 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "a59f85a20fd745e497d40788d30fe0dc"
format_stamp: "Formatted at 2026-08-10 06:45:00 on dist-test-slave-zj8q"
server_key: "b0e8ee3a4d90a36d98f27354148a5871"
server_key_iv: "e6c768ca7e4aa3c0d7441593aae04d36"
server_key_version: "encryptionkey@0"
I20260810 06:45:00.500648 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "a59f85a20fd745e497d40788d30fe0dc"
format_stamp: "Formatted at 2026-08-10 06:45:00 on dist-test-slave-zj8q"
server_key: "b0e8ee3a4d90a36d98f27354148a5871"
server_key_iv: "e6c768ca7e4aa3c0d7441593aae04d36"
server_key_version: "encryptionkey@0"
I20260810 06:45:00.505483 27584 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:45:00.508795 27608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:00.510023 27584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:45:00.510200 27584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "a59f85a20fd745e497d40788d30fe0dc"
format_stamp: "Formatted at 2026-08-10 06:45:00 on dist-test-slave-zj8q"
server_key: "b0e8ee3a4d90a36d98f27354148a5871"
server_key_iv: "e6c768ca7e4aa3c0d7441593aae04d36"
server_key_version: "encryptionkey@0"
I20260810 06:45:00.510402 27584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:00.515772 27584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:00.516494 27584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:00.516742 27584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:00.517577 27584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:00.518930 27584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:00.519019 27584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:00.519124 27584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:00.519198 27584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:00.548766 27584 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:46701
I20260810 06:45:00.548833 27787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:46701 every 8 connection(s)
I20260810 06:45:00.550058 27584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:00.557930 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:00.552533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:43425 (local address 127.10.242.62:35159)
0810 06:45:00.552966 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:00.552970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:00.554136 (+ 1166us) server_negotiation.cc:407] Connection header received
0810 06:45:00.555817 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:00.555823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:00.555884 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:00.555979 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:00.557177 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:00.557182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:00.557187 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:00.557204 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:00.557251 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:00.557264 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:00.557371 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:00.557553 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 06:45:00.557623 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":239,"threads_started":1}
I20260810 06:45:00.558826 27794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:00.553098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:35159 (local address 127.10.242.1:43425)
0810 06:45:00.553979 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:00.554015 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:00.555556 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:00.556234 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:00.556242 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:00.556327 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:00.556896 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:00.556898 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:00.556935 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:00.557017 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:00.557355 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:00.557366 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:00.557373 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:45:00.557621 (+ 248us) client_negotiation.cc:240] Negotiation successful
0810 06:45:00.557861 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":260,"threads_started":1}
I20260810 06:45:00.559196 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 27584
I20260810 06:45:00.559402 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:00.560123 11208 external_mini_cluster.cc:1428] Setting key 9ac2c41067ba8947b2d8597e3ea0725b
I20260810 06:45:00.560288 27788 heartbeater.cc:347] Connected to a master server at 127.10.242.62:35159
I20260810 06:45:00.560559 27788 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:00.561262 27788 heartbeater.cc:511] Master 127.10.242.62:35159 requested a full tablet report, sending...
I20260810 06:45:00.562968 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:35159
--builtin_ntp_servers=127.10.242.20:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 06:45:00.563371 27483 ts_manager.cc:194] Registered new tserver with Master: a59f85a20fd745e497d40788d30fe0dc (127.10.242.1:46701)
W20260810 06:45:00.886101 27797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:00.886379 27797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:00.886510 27797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:00.893412 27797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:00.893501 27797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:00.893656 27797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:45:00.902416 27797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:35159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:00.903688 27797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:00.905408 27797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:00.917742 27804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:00.917660 27809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:00.917714 27812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:00.918524 27797 server_base.cc:1034] running on GCE node
I20260810 06:45:00.919100 27797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:00.919919 27797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:00.921104 27797 hybrid_clock.cc:648] HybridClock initialized: now 1786344300921051 us; error 42 us; skew 500 ppm
I20260810 06:45:00.921378 27797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:45:00.923454 27797 webserver.cc:460] Webserver started at http://127.10.242.2:32775/ using document root <none> and password file <none>
I20260810 06:45:00.924065 27797 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:00.924154 27797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:00.924420 27797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:00.927155 27797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "760ec6c2c8c34ca388d3dc2a47d0328e"
format_stamp: "Formatted at 2026-08-10 06:45:00 on dist-test-slave-zj8q"
server_key: "e132666a0652b82bd98383c81c6d0bb8"
server_key_iv: "1a899a8cb951b00a15e1744c0a382155"
server_key_version: "encryptionkey@0"
I20260810 06:45:00.927876 27797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "760ec6c2c8c34ca388d3dc2a47d0328e"
format_stamp: "Formatted at 2026-08-10 06:45:00 on dist-test-slave-zj8q"
server_key: "e132666a0652b82bd98383c81c6d0bb8"
server_key_iv: "1a899a8cb951b00a15e1744c0a382155"
server_key_version: "encryptionkey@0"
I20260810 06:45:00.932569 27797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:45:00.935760 27823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:00.936972 27797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:45:00.937103 27797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "760ec6c2c8c34ca388d3dc2a47d0328e"
format_stamp: "Formatted at 2026-08-10 06:45:00 on dist-test-slave-zj8q"
server_key: "e132666a0652b82bd98383c81c6d0bb8"
server_key_iv: "1a899a8cb951b00a15e1744c0a382155"
server_key_version: "encryptionkey@0"
I20260810 06:45:00.937284 27797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:00.942520 27797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:00.943270 27797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:00.943511 27797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:00.944417 27797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:00.945752 27797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:00.945842 27797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:00.945947 27797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:00.946014 27797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:00.976397 27797 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:45105
I20260810 06:45:00.976460 28031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:45105 every 8 connection(s)
I20260810 06:45:00.977638 27797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:00.978255 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 27797
I20260810 06:45:00.978433 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:00.979003 11208 external_mini_cluster.cc:1428] Setting key cb184c402c789201f3a9a9e236472192
I20260810 06:45:00.981796 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:35159
--builtin_ntp_servers=127.10.242.20:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 06:45:00.985836 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:00.980142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:52831 (local address 127.10.242.62:35159)
0810 06:45:00.980307 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:00.980311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:00.981978 (+ 1667us) server_negotiation.cc:407] Connection header received
0810 06:45:00.983734 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:00.983739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:00.983793 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:00.983876 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:00.985094 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:00.985100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:00.985105 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:00.985123 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:00.985159 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:00.985171 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:00.985269 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:00.985459 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 06:45:00.985538 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:45:00.986713 28036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:00.980821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:35159 (local address 127.10.242.2:52831)
0810 06:45:00.981828 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:00.981872 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:00.983445 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:00.984119 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:00.984136 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:00.984220 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:00.984811 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:00.984814 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:00.984850 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:00.984932 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:00.985258 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:00.985269 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:00.985278 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:45:00.985538 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 06:45:00.985785 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":314,"threads_started":1}
I20260810 06:45:00.987955 28032 heartbeater.cc:347] Connected to a master server at 127.10.242.62:35159
I20260810 06:45:00.988240 28032 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:00.988785 28032 heartbeater.cc:511] Master 127.10.242.62:35159 requested a full tablet report, sending...
I20260810 06:45:00.990072 27483 ts_manager.cc:194] Registered new tserver with Master: 760ec6c2c8c34ca388d3dc2a47d0328e (127.10.242.2:45105)
W20260810 06:45:01.312199 28037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:01.312458 28037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:01.312585 28037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:01.319751 28037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:01.319846 28037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:01.319979 28037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:45:01.328799 28037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:35159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:01.330030 28037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:01.331893 28037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:01.344643 28044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:01.344661 28047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:01.345294 28043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:01.345623 28037 server_base.cc:1034] running on GCE node
I20260810 06:45:01.346237 28037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:01.347128 28037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:01.348333 28037 hybrid_clock.cc:648] HybridClock initialized: now 1786344301348287 us; error 33 us; skew 500 ppm
I20260810 06:45:01.348642 28037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:45:01.350811 28037 webserver.cc:460] Webserver started at http://127.10.242.3:41899/ using document root <none> and password file <none>
I20260810 06:45:01.351471 28037 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:01.351575 28037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:01.351886 28037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:01.354730 28037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "5f6f6a7a1561461e889ec6af07d05d8e"
format_stamp: "Formatted at 2026-08-10 06:45:01 on dist-test-slave-zj8q"
server_key: "9cef39e725d28bf15c0d089448fca539"
server_key_iv: "ab6d700f0eecef50acb6236d467d5f8d"
server_key_version: "encryptionkey@0"
I20260810 06:45:01.355499 28037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "5f6f6a7a1561461e889ec6af07d05d8e"
format_stamp: "Formatted at 2026-08-10 06:45:01 on dist-test-slave-zj8q"
server_key: "9cef39e725d28bf15c0d089448fca539"
server_key_iv: "ab6d700f0eecef50acb6236d467d5f8d"
server_key_version: "encryptionkey@0"
I20260810 06:45:01.360330 28037 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:45:01.363606 28057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:01.364851 28037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:45:01.365007 28037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "5f6f6a7a1561461e889ec6af07d05d8e"
format_stamp: "Formatted at 2026-08-10 06:45:01 on dist-test-slave-zj8q"
server_key: "9cef39e725d28bf15c0d089448fca539"
server_key_iv: "ab6d700f0eecef50acb6236d467d5f8d"
server_key_version: "encryptionkey@0"
I20260810 06:45:01.365201 28037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:01.370769 28037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:01.371518 28037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:01.371766 28037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:01.372673 28037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:01.374145 28037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:01.374280 28037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:01.374385 28037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:01.374454 28037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:01.404241 28037 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:40545
I20260810 06:45:01.404284 28246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:40545 every 8 connection(s)
I20260810 06:45:01.405646 28037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:01.407111 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 28037
I20260810 06:45:01.407331 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:01.407886 11208 external_mini_cluster.cc:1428] Setting key b6c513cd0ff8a1db762722be62d68f13
I20260810 06:45:01.415736 27792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:01.410025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:43413 (local address 127.10.242.62:35159)
0810 06:45:01.410265 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:01.410270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:01.411892 (+ 1622us) server_negotiation.cc:407] Connection header received
0810 06:45:01.413579 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:01.413584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:01.413652 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:01.413745 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:01.415043 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:01.415049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:01.415054 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:01.415074 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:01.415106 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:01.415118 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:01.415230 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:01.415406 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 06:45:01.415472 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:45:01.416592 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:01.410774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:35159 (local address 127.10.242.3:43413)
0810 06:45:01.411745 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:01.411792 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:01.413301 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:01.413980 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:01.413987 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:01.414072 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:01.414753 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:45:01.414755 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:01.414792 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:45:01.414879 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:01.415211 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:01.415221 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:01.415229 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:45:01.415478 (+ 249us) client_negotiation.cc:240] Negotiation successful
0810 06:45:01.415735 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":315,"threads_started":1}
I20260810 06:45:01.417752 28249 heartbeater.cc:347] Connected to a master server at 127.10.242.62:35159
I20260810 06:45:01.417997 28249 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:01.418593 28249 heartbeater.cc:511] Master 127.10.242.62:35159 requested a full tablet report, sending...
I20260810 06:45:01.419860 27483 ts_manager.cc:194] Registered new tserver with Master: 5f6f6a7a1561461e889ec6af07d05d8e (127.10.242.3:40545)
I20260810 06:45:01.423333 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:45:01.429214 27792 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:45:01.427710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49066 (local address 127.10.242.62:35159)
0810 06:45:01.427856 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:01.427860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:01.428367 (+ 507us) server_negotiation.cc:407] Connection header received
0810 06:45:01.428498 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:01.428501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:01.428549 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:01.428617 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:01.428969 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49066: BlockingRecv error: recv got EOF from 127.0.0.1:49066 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:45:01.429909 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 27584
I20260810 06:45:01.433066 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:01.430822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43756 (local address 127.10.242.1:46701)
0810 06:45:01.431303 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:01.431309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:01.431332 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:45:01.431388 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:01.431393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:01.431562 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:01.431676 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:01.432464 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:01.432469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:01.432478 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:01.432495 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:01.432531 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:01.432544 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:01.432647 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:01.432696 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 06:45:01.432768 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":227,"threads_started":1}
I20260810 06:45:01.434237 27783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:01.512943 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 27584
I20260810 06:45:01.537259 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 27797
I20260810 06:45:01.540779 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:01.538211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.10.242.2:45105)
0810 06:45:01.538748 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:01.538755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:01.538782 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:45:01.538848 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:01.538854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:01.539070 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:01.539201 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:01.540078 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:01.540087 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:01.540103 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:01.540128 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:01.540175 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:01.540191 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:01.540295 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:01.540374 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:45:01.540465 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":268,"threads_started":1}
I20260810 06:45:01.541981 28019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:01.621070 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 27797
I20260810 06:45:01.646431 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 28037
I20260810 06:45:01.650405 28271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:01.647382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.10.242.3:40545)
0810 06:45:01.648034 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:01.648041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:01.648070 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:45:01.648135 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:01.648141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:01.648367 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:01.648507 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:01.649616 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:01.649622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:01.649634 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:45:01.649650 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:01.649690 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:45:01.649704 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:01.649855 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:01.649981 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 06:45:01.650086 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":343,"threads_started":1}
I20260810 06:45:01.651587 28241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:01.730558 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 28037
I20260810 06:45:01.753494 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 27437
I20260810 06:45:01.754751 27524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:01.855782 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 27437
2026-08-10T06:45:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2318 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:01 dist-test-slave-zj8q krb5kdc[28282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:01 dist-test-slave-zj8q krb5kdc[28282](info): set up 2 sockets
Aug 10 06:45:01 dist-test-slave-zj8q krb5kdc[28282](info): commencing operation
krb5kdc: starting...
W20260810 06:45:03.094282 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:45:03 dist-test-slave-zj8q krb5kdc[28282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:45:03.354876 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:45951
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:03.684890 28303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:03.685189 28303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:03.685326 28303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:03.692545 28303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:03.692659 28303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:03.692740 28303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:03.692830 28303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:03.692895 28303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:03.702595 28303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:45951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:03.703833 28303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:03.705140 28303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:03.716578 28312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:03.716540 28310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:03.716610 28321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:03.717250 28303 server_base.cc:1034] running on GCE node
I20260810 06:45:03.718053 28303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:03.719384 28303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:03.720623 28303 hybrid_clock.cc:648] HybridClock initialized: now 1786344303720577 us; error 28 us; skew 500 ppm
I20260810 06:45:03.720947 28303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:45:03 dist-test-slave-zj8q krb5kdc[28282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:03.727486 28303 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:45:03.728960 28303 webserver.cc:460] Webserver started at http://127.10.242.62:46335/ using document root <none> and password file <none>
I20260810 06:45:03.729622 28303 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:03.729736 28303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:03.730036 28303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:03.733012 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "36fb283b1c1446bcba9295e7a4f14f43"
format_stamp: "Formatted at 2026-08-10 06:45:03 on dist-test-slave-zj8q"
server_key: "ceceff35325bdbcf66f0120fb334138a"
server_key_iv: "f0e58e36e7715af06423b4ce17540b95"
server_key_version: "encryptionkey@0"
I20260810 06:45:03.733774 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "36fb283b1c1446bcba9295e7a4f14f43"
format_stamp: "Formatted at 2026-08-10 06:45:03 on dist-test-slave-zj8q"
server_key: "ceceff35325bdbcf66f0120fb334138a"
server_key_iv: "f0e58e36e7715af06423b4ce17540b95"
server_key_version: "encryptionkey@0"
I20260810 06:45:03.738772 28303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 06:45:03.742081 28329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:03.743433 28303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:45:03.743604 28303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "36fb283b1c1446bcba9295e7a4f14f43"
format_stamp: "Formatted at 2026-08-10 06:45:03 on dist-test-slave-zj8q"
server_key: "ceceff35325bdbcf66f0120fb334138a"
server_key_iv: "f0e58e36e7715af06423b4ce17540b95"
server_key_version: "encryptionkey@0"
I20260810 06:45:03.743798 28303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:03.764599 28303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:03.768747 28303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:03.769014 28303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:03.786134 28303 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:45951
I20260810 06:45:03.786194 28419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:45951 every 8 connection(s)
I20260810 06:45:03.787446 28303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:03.788807 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 28303
I20260810 06:45:03.789044 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:45:03.789902 11208 external_mini_cluster.cc:1428] Setting key e4e4d51f1871f1e54cda3825991e39a0
I20260810 06:45:03.791935 28422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:45:03 dist-test-slave-zj8q krb5kdc[28282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:45:03.805123 28422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43: Bootstrap starting.
I20260810 06:45:03.808295 28422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:03.809504 28422 log.cc:824] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:03.812896 28422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43: No bootstrap required, opened a new log
I20260810 06:45:03.813150 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:03.791951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57376 (local address 127.10.242.62:45951)
0810 06:45:03.792736 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:03.792747 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:03.792789 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:45:03.794399 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:03.794421 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:03.794832 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:03.795162 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:03.796996 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:03.798438 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:03.800147 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:03.800804 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:03.800951 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:03.805735 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:03.805743 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:03.805757 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:03.805796 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:03.808710 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:03.809474 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:03.809480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:03.809486 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:03.809663 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:03.810094 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:03.810099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:03.810102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:03.810916 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:03.811224 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:03.811722 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 06:45:03.812150 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":346,"threads_started":1}
I20260810 06:45:03.820183 28422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fb283b1c1446bcba9295e7a4f14f43" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45951 } }
I20260810 06:45:03.820547 28422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:03.820649 28422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36fb283b1c1446bcba9295e7a4f14f43, State: Initialized, Role: FOLLOWER
I20260810 06:45:03.821162 28422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fb283b1c1446bcba9295e7a4f14f43" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45951 } }
I20260810 06:45:03.821381 28422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:03.821507 28422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:03.821672 28422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:03.823921 28422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fb283b1c1446bcba9295e7a4f14f43" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45951 } }
I20260810 06:45:03.824388 28422 leader_election.cc:302] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36fb283b1c1446bcba9295e7a4f14f43; no voters:
I20260810 06:45:03.824965 28422 leader_election.cc:288] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:03.825076 28431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:03.825793 28431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [term 1 LEADER]: Becoming Leader. State: Replica: 36fb283b1c1446bcba9295e7a4f14f43, State: Running, Role: LEADER
I20260810 06:45:03.826436 28431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fb283b1c1446bcba9295e7a4f14f43" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45951 } }
I20260810 06:45:03.827044 28422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:03.829677 28432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36fb283b1c1446bcba9295e7a4f14f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fb283b1c1446bcba9295e7a4f14f43" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45951 } } }
I20260810 06:45:03.829749 28434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36fb283b1c1446bcba9295e7a4f14f43. Latest consensus state: current_term: 1 leader_uuid: "36fb283b1c1446bcba9295e7a4f14f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36fb283b1c1446bcba9295e7a4f14f43" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45951 } } }
I20260810 06:45:03.830145 28432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:03.830178 28434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43 [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:03.831146 28440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:03.836923 28440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:03.846441 28440 catalog_manager.cc:1347] Generated new cluster ID: b0a3554cfafb40119968e5e4dd042a43
I20260810 06:45:03.846547 28440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:03.859814 28440 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:03.860998 28440 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:03.877465 28440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43: Generated new TSK 0
I20260810 06:45:03.878273 28440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:45:03.964816 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45951
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:04.309829 28467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:04.310111 28467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:04.310266 28467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:04.318051 28467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:04.318190 28467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:04.318333 28467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:45:04.328161 28467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:04.329586 28467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:04.330919 28467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:04.344062 28477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:04.344133 28481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:04.344154 28479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:04.345166 28467 server_base.cc:1034] running on GCE node
I20260810 06:45:04.345805 28467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:04.346738 28467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:04.347927 28467 hybrid_clock.cc:648] HybridClock initialized: now 1786344304347882 us; error 37 us; skew 500 ppm
I20260810 06:45:04.348277 28467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:45:04 dist-test-slave-zj8q krb5kdc[28282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344304, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:04.351460 28467 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:45:04.352818 28467 webserver.cc:460] Webserver started at http://127.10.242.1:34205/ using document root <none> and password file <none>
I20260810 06:45:04.353475 28467 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:04.353598 28467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:04.353881 28467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:04.356767 28467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "43217e1ea4f14270a70f12ab0c2a0d49"
format_stamp: "Formatted at 2026-08-10 06:45:04 on dist-test-slave-zj8q"
server_key: "d291359109125c5462221ff9ac3285bf"
server_key_iv: "3b2a33c1aabfe4c73e207f7e834db5e7"
server_key_version: "encryptionkey@0"
I20260810 06:45:04.357492 28467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "43217e1ea4f14270a70f12ab0c2a0d49"
format_stamp: "Formatted at 2026-08-10 06:45:04 on dist-test-slave-zj8q"
server_key: "d291359109125c5462221ff9ac3285bf"
server_key_iv: "3b2a33c1aabfe4c73e207f7e834db5e7"
server_key_version: "encryptionkey@0"
I20260810 06:45:04.362425 28467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 06:45:04.365641 28489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:04.366966 28467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:04.367106 28467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "43217e1ea4f14270a70f12ab0c2a0d49"
format_stamp: "Formatted at 2026-08-10 06:45:04 on dist-test-slave-zj8q"
server_key: "d291359109125c5462221ff9ac3285bf"
server_key_iv: "3b2a33c1aabfe4c73e207f7e834db5e7"
server_key_version: "encryptionkey@0"
I20260810 06:45:04.367269 28467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:04.384673 28467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:04.388468 28467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:04.388721 28467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:04.389592 28467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:04.390882 28467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:04.390959 28467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:04.391060 28467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:04.391103 28467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:04.421355 28467 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:36211
I20260810 06:45:04.421401 28687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:36211 every 8 connection(s)
I20260810 06:45:04.422609 28467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:04.427320 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 28467
I20260810 06:45:04.427538 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:04.428347 11208 external_mini_cluster.cc:1428] Setting key f8bb1fbb2338767e480835d38618af95
Aug 10 06:45:04 dist-test-slave-zj8q krb5kdc[28282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344304, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:45:04.443661 28692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:04.425075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:46599 (local address 127.10.242.62:45951)
0810 06:45:04.425537 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:04.425542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:04.426490 (+ 948us) server_negotiation.cc:407] Connection header received
0810 06:45:04.428923 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:04.428930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:04.429018 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:04.429145 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:04.430981 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:04.431940 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:04.434067 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:04.434522 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:04.434638 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:04.438266 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:04.438271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:04.438274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:04.438307 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:04.440712 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:04.441384 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:04.441390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:04.441393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:04.441514 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:04.441933 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:04.441937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:04.441941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:04.442257 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:04.442391 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:04.443075 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 06:45:04.443308 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":214,"threads_started":1}
I20260810 06:45:04.444664 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:04.425609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:45951 (local address 127.10.242.1:46599)
0810 06:45:04.426352 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:04.426390 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:04.428637 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:04.429396 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:04.429408 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:04.429978 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:04.430777 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:04.430803 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:04.432124 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:04.432130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:04.433902 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:04.433918 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:04.434741 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:04.434749 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:04.435006 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:04.435647 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:04.435679 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:04.438040 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:04.440843 (+ 2803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:04.440851 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:04.440866 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:04.441271 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:04.441618 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:04.441631 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:04.441635 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:04.441826 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:04.442377 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:04.442385 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:04.442709 (+ 324us) client_negotiation.cc:769] Sending connection context
0810 06:45:04.442950 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 06:45:04.443435 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":277,"threads_started":1}
I20260810 06:45:04.446223 28689 heartbeater.cc:347] Connected to a master server at 127.10.242.62:45951
I20260810 06:45:04.446496 28689 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:04.447252 28689 heartbeater.cc:511] Master 127.10.242.62:45951 requested a full tablet report, sending...
I20260810 06:45:04.449460 28354 ts_manager.cc:194] Registered new tserver with Master: 43217e1ea4f14270a70f12ab0c2a0d49 (127.10.242.1:36211)
I20260810 06:45:04.451483 28354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:46599
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:45:04.488696 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45951
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:04.812016 28699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:04.812271 28699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:04.812364 28699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:04.819368 28699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:04.819471 28699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:04.819592 28699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:45:04.829221 28699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:04.830539 28699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:04.831790 28699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:04.844444 28709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:04.844465 28710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:04.845054 28713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:04.845126 28699 server_base.cc:1034] running on GCE node
I20260810 06:45:04.845824 28699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:04.846738 28699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:04.847939 28699 hybrid_clock.cc:648] HybridClock initialized: now 1786344304847885 us; error 43 us; skew 500 ppm
I20260810 06:45:04.848248 28699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:45:04 dist-test-slave-zj8q krb5kdc[28282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344304, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:04.851406 28699 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:45:04.852908 28699 webserver.cc:460] Webserver started at http://127.10.242.2:44547/ using document root <none> and password file <none>
I20260810 06:45:04.853554 28699 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:04.853682 28699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:04.853963 28699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:04.856819 28699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "aee4cd1825fb42a7a16d5341cacc419e"
format_stamp: "Formatted at 2026-08-10 06:45:04 on dist-test-slave-zj8q"
server_key: "06062a0c9b1ff3699968f414eb9ed7e3"
server_key_iv: "ba43ad83162555f9f30c1f1fc1ef115b"
server_key_version: "encryptionkey@0"
I20260810 06:45:04.857520 28699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "aee4cd1825fb42a7a16d5341cacc419e"
format_stamp: "Formatted at 2026-08-10 06:45:04 on dist-test-slave-zj8q"
server_key: "06062a0c9b1ff3699968f414eb9ed7e3"
server_key_iv: "ba43ad83162555f9f30c1f1fc1ef115b"
server_key_version: "encryptionkey@0"
I20260810 06:45:04.862360 28699 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.002s
I20260810 06:45:04.865588 28723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:04.866879 28699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:45:04.867004 28699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "aee4cd1825fb42a7a16d5341cacc419e"
format_stamp: "Formatted at 2026-08-10 06:45:04 on dist-test-slave-zj8q"
server_key: "06062a0c9b1ff3699968f414eb9ed7e3"
server_key_iv: "ba43ad83162555f9f30c1f1fc1ef115b"
server_key_version: "encryptionkey@0"
I20260810 06:45:04.867154 28699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:04.882884 28699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:04.886684 28699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:04.886924 28699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:04.887758 28699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:04.888988 28699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:04.889060 28699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:04.889153 28699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:04.889195 28699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:04.918859 28699 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:45907
I20260810 06:45:04.918902 28903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:45907 every 8 connection(s)
I20260810 06:45:04.920079 28699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:04.921574 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 28699
I20260810 06:45:04.921788 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:04.922360 11208 external_mini_cluster.cc:1428] Setting key 2c2c0026b135d943b342de3ec1b4fdc9
Aug 10 06:45:04 dist-test-slave-zj8q krb5kdc[28282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344304, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:45:04.940358 28692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:04.922791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:51663 (local address 127.10.242.62:45951)
0810 06:45:04.923013 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:04.923020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:04.924191 (+ 1171us) server_negotiation.cc:407] Connection header received
0810 06:45:04.926199 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:04.926206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:04.926288 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:04.926394 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:04.927913 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:04.928753 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:04.930304 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:04.930750 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:04.930864 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:04.934313 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:04.934320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:04.934325 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:04.934359 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:04.937183 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:04.938033 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:04.938037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:04.938039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:04.938211 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:04.938698 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:04.938701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:04.938703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:04.938959 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:04.939088 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:04.939917 (+ 829us) server_negotiation.cc:299] Negotiation successful
0810 06:45:04.940093 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:45:04.941749 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:04.923237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:45951 (local address 127.10.242.2:51663)
0810 06:45:04.924033 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:04.924083 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:04.925909 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:04.926587 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:04.926595 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:04.927160 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:04.927744 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:04.927763 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:04.928912 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:04.928916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:04.930138 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:04.930180 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:04.930971 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:04.930979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:04.931215 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:04.931822 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:04.931844 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:04.934062 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:04.937347 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:04.937354 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:04.937366 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:04.937915 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:04.938314 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:04.938320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:04.938324 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:04.938592 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:04.939087 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:04.939095 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:04.939507 (+ 412us) client_negotiation.cc:769] Sending connection context
0810 06:45:04.939824 (+ 317us) client_negotiation.cc:240] Negotiation successful
0810 06:45:04.940403 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":297,"threads_started":1}
I20260810 06:45:04.943305 28906 heartbeater.cc:347] Connected to a master server at 127.10.242.62:45951
I20260810 06:45:04.943588 28906 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:04.944332 28906 heartbeater.cc:511] Master 127.10.242.62:45951 requested a full tablet report, sending...
I20260810 06:45:04.945775 28354 ts_manager.cc:194] Registered new tserver with Master: aee4cd1825fb42a7a16d5341cacc419e (127.10.242.2:45907)
I20260810 06:45:04.946664 28354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:51663
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:45:04.986924 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45951
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:05.317332 28915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:05.317601 28915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:05.317706 28915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:05.325011 28915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:05.325120 28915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:05.325232 28915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:45:05.335467 28915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:05.336760 28915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:05.338055 28915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:05.350963 28924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.350915 28926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:05.350986 28923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:05.351734 28915 server_base.cc:1034] running on GCE node
I20260810 06:45:05.352352 28915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:05.353191 28915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:05.354417 28915 hybrid_clock.cc:648] HybridClock initialized: now 1786344305354363 us; error 44 us; skew 500 ppm
I20260810 06:45:05.354727 28915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:45:05 dist-test-slave-zj8q krb5kdc[28282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:05.357630 28915 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:45:05.359054 28915 webserver.cc:460] Webserver started at http://127.10.242.3:36479/ using document root <none> and password file <none>
I20260810 06:45:05.359704 28915 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:05.359820 28915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:05.360097 28915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:45:05.362901 28915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "1382721754944169a874a04716552a81"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-zj8q"
server_key: "ec4e0209429e9f29d94cdf50abb529d8"
server_key_iv: "1766feb1cd6b4296797b85899bb2ff4e"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.363612 28915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "1382721754944169a874a04716552a81"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-zj8q"
server_key: "ec4e0209429e9f29d94cdf50abb529d8"
server_key_iv: "1766feb1cd6b4296797b85899bb2ff4e"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.368534 28915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:45:05.371974 28935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.373239 28915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:45:05.373386 28915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "1382721754944169a874a04716552a81"
format_stamp: "Formatted at 2026-08-10 06:45:05 on dist-test-slave-zj8q"
server_key: "ec4e0209429e9f29d94cdf50abb529d8"
server_key_iv: "1766feb1cd6b4296797b85899bb2ff4e"
server_key_version: "encryptionkey@0"
I20260810 06:45:05.373574 28915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:05.394685 28915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:05.398748 28915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:05.399009 28915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:05.399960 28915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:05.401229 28915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:05.401310 28915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.401425 28915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:05.401475 28915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:05.432653 28915 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:38009
I20260810 06:45:05.432693 29121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:38009 every 8 connection(s)
I20260810 06:45:05.433811 28915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:05.442549 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 28915
I20260810 06:45:05.442732 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:05.443125 11208 external_mini_cluster.cc:1428] Setting key c664282368b4b503f366f57a819f03f2
Aug 10 06:45:05 dist-test-slave-zj8q krb5kdc[28282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344305, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:45:05.452765 28692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.436241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:45919 (local address 127.10.242.62:45951)
0810 06:45:05.436406 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.436410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.437522 (+ 1112us) server_negotiation.cc:407] Connection header received
0810 06:45:05.439346 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.439352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.439435 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.439521 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:05.440960 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.441702 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.443339 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.443834 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.443954 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.447450 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:05.447455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:05.447459 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:05.447493 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:05.449905 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.450589 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.450593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.450595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.450733 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.451066 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.451069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.451071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.451316 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:05.451448 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.452205 (+ 757us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.452419 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:45:05.453652 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.436692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:45951 (local address 127.10.242.3:45919)
0810 06:45:05.437370 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:05.437413 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:05.439074 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:05.439762 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:05.439769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:05.440256 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:05.440788 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.440805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.441844 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.441848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.443200 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.443217 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.444077 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.444085 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.444337 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.444965 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:05.444991 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:05.447267 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:05.450027 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:05.450034 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:05.450045 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:05.450494 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:05.450813 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:05.450816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:05.450818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:05.450987 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:05.451463 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:05.451471 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:05.451834 (+ 363us) client_negotiation.cc:769] Sending connection context
0810 06:45:05.452063 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 06:45:05.452564 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":248,"threads_started":1}
I20260810 06:45:05.454952 29122 heartbeater.cc:347] Connected to a master server at 127.10.242.62:45951
I20260810 06:45:05.454955 28689 heartbeater.cc:503] Master 127.10.242.62:45951 was elected leader, sending a full tablet report...
I20260810 06:45:05.455238 29122 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:05.455917 29122 heartbeater.cc:511] Master 127.10.242.62:45951 requested a full tablet report, sending...
I20260810 06:45:05.457381 28354 ts_manager.cc:194] Registered new tserver with Master: 1382721754944169a874a04716552a81 (127.10.242.3:38009)
I20260810 06:45:05.458288 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:45:05.458325 28354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:45919
I20260810 06:45:05.474514 28692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.462650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57390 (local address 127.10.242.62:45951)
0810 06:45:05.462794 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.462798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.463283 (+ 485us) server_negotiation.cc:407] Connection header received
0810 06:45:05.463438 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.463443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.463517 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.463600 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:05.464958 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.465652 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.466926 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.467336 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.467440 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.469459 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:05.469464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:05.469467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:05.469497 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:05.472051 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.472630 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.472636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.472640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.472764 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.473109 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.473114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.473118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.473434 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:05.473692 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.473985 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.474213 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:45:05.478089 28354 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:05.484759 28354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:05.506742 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.496736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37204 (local address 127.10.242.1:36211)
0810 06:45:05.497505 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.497513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.498062 (+ 549us) server_negotiation.cc:407] Connection header received
0810 06:45:05.498481 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.498489 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.498735 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.498888 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:05.499977 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.501338 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.504115 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.505586 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.505734 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.505952 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.506238 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.506401 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":444,"threads_started":1}
I20260810 06:45:05.506805 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.496147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:36211 (local address 127.0.0.1:37204)
0810 06:45:05.497892 (+ 1745us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:05.497960 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:05.498239 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:05.499201 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:05.499209 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:05.499256 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:05.499756 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.499771 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.501490 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.501495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.503941 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.503956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.505787 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.505794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.505980 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.506031 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:45:05.506188 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 06:45:05.506372 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1632,"mutex_wait_us":52,"spinlock_wait_cycles":29824,"thread_start_us":1501,"threads_started":1}
I20260810 06:45:05.507071 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.496224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.10.242.2:45907)
0810 06:45:05.497035 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.497043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.498067 (+ 1024us) server_negotiation.cc:407] Connection header received
0810 06:45:05.498481 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.498489 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.498723 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.498893 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:05.500588 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.502061 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.504433 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.505735 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.505869 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.506052 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.506565 (+ 513us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.506741 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":715,"threads_started":1}
I20260810 06:45:05.507989 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.498618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.10.242.3:38009)
0810 06:45:05.499968 (+ 1350us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.499975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.500359 (+ 384us) server_negotiation.cc:407] Connection header received
0810 06:45:05.500537 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.500545 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.500780 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.500986 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:05.501932 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.503048 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.505311 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.506697 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.506808 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.507017 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.507521 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.507673 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1184,"thread_start_us":1010,"threads_started":1}
I20260810 06:45:05.507995 29145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.495953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:45907 (local address 127.0.0.1:36992)
0810 06:45:05.497892 (+ 1939us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:05.497962 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:05.498239 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:05.499201 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:05.499209 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:05.499245 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:05.500355 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.500369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.502221 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.502230 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.504294 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.504305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.506147 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.506177 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.506348 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.506380 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:45:05.507400 (+ 1020us) client_negotiation.cc:240] Negotiation successful
0810 06:45:05.507552 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1670,"mutex_wait_us":605,"spinlock_wait_cycles":97536,"thread_start_us":1525,"threads_started":1}
I20260810 06:45:05.508090 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.498970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38009 (local address 127.0.0.1:59222)
0810 06:45:05.500245 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:05.500259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:05.500386 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:05.501252 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:05.501257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:05.501283 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:05.501730 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.501742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.503188 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.503192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.505111 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.505136 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.507175 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.507182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.507339 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.507370 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:45:05.507461 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:45:05.507595 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1152,"mutex_wait_us":64,"thread_start_us":1003,"threads_started":1}
I20260810 06:45:05.510589 28574 tablet_service.cc:1467] Processing CreateTablet for tablet cd1b158641074389bd01bc93e364d663 (DEFAULT_TABLE table=test-table [id=42db36c14ced4a3c8cb8abd1ae7f3c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:05.510987 28804 tablet_service.cc:1467] Processing CreateTablet for tablet cd1b158641074389bd01bc93e364d663 (DEFAULT_TABLE table=test-table [id=42db36c14ced4a3c8cb8abd1ae7f3c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:05.511998 28574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1b158641074389bd01bc93e364d663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:05.512061 28804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1b158641074389bd01bc93e364d663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:05.513571 29018 tablet_service.cc:1467] Processing CreateTablet for tablet cd1b158641074389bd01bc93e364d663 (DEFAULT_TABLE table=test-table [id=42db36c14ced4a3c8cb8abd1ae7f3c11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:05.514900 29018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd1b158641074389bd01bc93e364d663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:05.522951 29152 tablet_bootstrap.cc:492] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: Bootstrap starting.
I20260810 06:45:05.525750 29153 tablet_bootstrap.cc:492] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: Bootstrap starting.
I20260810 06:45:05.526700 29152 tablet_bootstrap.cc:654] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:05.527994 29152 log.cc:824] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:05.528258 29155 tablet_bootstrap.cc:492] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: Bootstrap starting.
I20260810 06:45:05.529332 29153 tablet_bootstrap.cc:654] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:05.530459 29153 log.cc:824] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:05.531286 29155 tablet_bootstrap.cc:654] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:05.531594 29152 tablet_bootstrap.cc:492] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: No bootstrap required, opened a new log
I20260810 06:45:05.531857 29152 ts_tablet_manager.cc:1397] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 06:45:05.532318 29155 log.cc:824] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:05.533483 29153 tablet_bootstrap.cc:492] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: No bootstrap required, opened a new log
I20260810 06:45:05.533735 29153 ts_tablet_manager.cc:1397] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 06:45:05.534693 29155 tablet_bootstrap.cc:492] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: No bootstrap required, opened a new log
I20260810 06:45:05.535161 29155 ts_tablet_manager.cc:1397] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 06:45:05.542016 29155 raft_consensus.cc:348] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } }
I20260810 06:45:05.541935 29152 raft_consensus.cc:348] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } }
I20260810 06:45:05.542387 29155 raft_consensus.cc:374] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:05.542384 29152 raft_consensus.cc:374] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:05.542475 29155 raft_consensus.cc:729] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1382721754944169a874a04716552a81, State: Initialized, Role: FOLLOWER
I20260810 06:45:05.542476 29152 raft_consensus.cc:729] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aee4cd1825fb42a7a16d5341cacc419e, State: Initialized, Role: FOLLOWER
I20260810 06:45:05.542996 29155 consensus_queue.cc:260] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } }
I20260810 06:45:05.543102 29152 consensus_queue.cc:260] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } }
I20260810 06:45:05.544030 29122 heartbeater.cc:503] Master 127.10.242.62:45951 was elected leader, sending a full tablet report...
I20260810 06:45:05.544528 28906 heartbeater.cc:503] Master 127.10.242.62:45951 was elected leader, sending a full tablet report...
I20260810 06:45:05.544634 29155 ts_tablet_manager.cc:1428] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20260810 06:45:05.543751 29153 raft_consensus.cc:348] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } }
I20260810 06:45:05.546358 29153 raft_consensus.cc:374] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:05.546463 29153 raft_consensus.cc:729] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43217e1ea4f14270a70f12ab0c2a0d49, State: Initialized, Role: FOLLOWER
I20260810 06:45:05.547075 29153 consensus_queue.cc:260] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } }
I20260810 06:45:05.548033 29152 ts_tablet_manager.cc:1428] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260810 06:45:05.548782 29153 ts_tablet_manager.cc:1428] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
W20260810 06:45:05.672874 28907 tablet.cc:2369] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:45:05.674736 28690 tablet.cc:2369] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:45:05.685307 29123 tablet.cc:2369] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:45:05.802328 29163 raft_consensus.cc:482] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:05.802623 29163 raft_consensus.cc:504] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } }
I20260810 06:45:05.804323 29163 leader_election.cc:288] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43217e1ea4f14270a70f12ab0c2a0d49 (127.10.242.1:36211), 1382721754944169a874a04716552a81 (127.10.242.3:38009)
I20260810 06:45:05.810958 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.804785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:36345 (local address 127.10.242.3:38009)
0810 06:45:05.804987 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.804993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.805608 (+ 615us) server_negotiation.cc:407] Connection header received
0810 06:45:05.805827 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.805831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.805893 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.805967 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:05.806634 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.807377 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.809259 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.810009 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.810186 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.810264 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.810595 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.810720 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:45:05.810972 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.804708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:56683 (local address 127.10.242.1:36211)
0810 06:45:05.804923 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.804931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.805714 (+ 783us) server_negotiation.cc:407] Connection header received
0810 06:45:05.805877 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.805881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.805942 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.806014 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:05.806668 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.807380 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.809247 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.810023 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.810190 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.810267 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.810608 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.810743 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:45:05.811167 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.804607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38009 (local address 127.10.242.2:36345)
0810 06:45:05.805475 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:05.805491 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:05.805659 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:05.806002 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:05.806007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:05.806032 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:05.806477 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.806487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.807505 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.807509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.809103 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.809116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.810263 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.810270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.810416 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.810467 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:45:05.810548 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:45:05.810679 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":75,"spinlock_wait_cycles":18816,"thread_start_us":488,"threads_started":1}
I20260810 06:45:05.811198 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.804607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:36211 (local address 127.10.242.2:56683)
0810 06:45:05.805583 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:05.805606 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:05.805743 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:05.806043 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:05.806047 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:05.806070 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:05.806509 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.806518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.807506 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.807511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.809088 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:05.809101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.810301 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.810306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:05.810465 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.810488 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:45:05.810566 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:45:05.810689 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":125,"thread_start_us":466,"threads_started":1}
I20260810 06:45:05.812274 29050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1b158641074389bd01bc93e364d663" candidate_uuid: "aee4cd1825fb42a7a16d5341cacc419e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1382721754944169a874a04716552a81" is_pre_election: true
I20260810 06:45:05.812275 28607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1b158641074389bd01bc93e364d663" candidate_uuid: "aee4cd1825fb42a7a16d5341cacc419e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" is_pre_election: true
I20260810 06:45:05.812834 29050 raft_consensus.cc:2454] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aee4cd1825fb42a7a16d5341cacc419e in term 0.
I20260810 06:45:05.812834 28607 raft_consensus.cc:2454] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aee4cd1825fb42a7a16d5341cacc419e in term 0.
I20260810 06:45:05.813548 28728 leader_election.cc:302] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1382721754944169a874a04716552a81, aee4cd1825fb42a7a16d5341cacc419e; no voters:
I20260810 06:45:05.813941 29163 raft_consensus.cc:2781] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:05.814075 29163 raft_consensus.cc:482] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:05.814199 29163 raft_consensus.cc:3037] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:05.816388 29163 raft_consensus.cc:504] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } }
I20260810 06:45:05.816938 29163 leader_election.cc:288] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [CANDIDATE]: Term 1 election: Requested vote from peers 43217e1ea4f14270a70f12ab0c2a0d49 (127.10.242.1:36211), 1382721754944169a874a04716552a81 (127.10.242.3:38009)
I20260810 06:45:05.817373 28607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1b158641074389bd01bc93e364d663" candidate_uuid: "aee4cd1825fb42a7a16d5341cacc419e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43217e1ea4f14270a70f12ab0c2a0d49"
I20260810 06:45:05.817410 29050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd1b158641074389bd01bc93e364d663" candidate_uuid: "aee4cd1825fb42a7a16d5341cacc419e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1382721754944169a874a04716552a81"
I20260810 06:45:05.817587 28607 raft_consensus.cc:3037] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:05.817589 29050 raft_consensus.cc:3037] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:05.819736 29050 raft_consensus.cc:2454] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aee4cd1825fb42a7a16d5341cacc419e in term 1.
I20260810 06:45:05.819767 28607 raft_consensus.cc:2454] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aee4cd1825fb42a7a16d5341cacc419e in term 1.
I20260810 06:45:05.820096 28728 leader_election.cc:302] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1382721754944169a874a04716552a81, aee4cd1825fb42a7a16d5341cacc419e; no voters:
I20260810 06:45:05.820363 29163 raft_consensus.cc:2781] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:05.820753 29163 raft_consensus.cc:686] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 1 LEADER]: Becoming Leader. State: Replica: aee4cd1825fb42a7a16d5341cacc419e, State: Running, Role: LEADER
I20260810 06:45:05.821208 29163 consensus_queue.cc:237] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } }
I20260810 06:45:05.825910 28351 catalog_manager.cc:5495] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e reported cstate change: term changed from 0 to 1, leader changed from <none> to aee4cd1825fb42a7a16d5341cacc419e (127.10.242.2). New cstate: current_term: 1 leader_uuid: "aee4cd1825fb42a7a16d5341cacc419e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee4cd1825fb42a7a16d5341cacc419e" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 } health_report { overall_health: UNKNOWN } } }
I20260810 06:45:05.870250 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.865181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36994 (local address 127.10.242.2:45907)
0810 06:45:05.865402 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.865408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.865427 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:45:05.865533 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.865537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.865621 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.865753 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:05.866130 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.866879 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.868168 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.868569 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.868676 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.869131 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:05.869626 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:05.869756 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.869861 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.869940 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:45:05.877918 29050 raft_consensus.cc:1261] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [term 1 FOLLOWER]: Refusing update from remote peer aee4cd1825fb42a7a16d5341cacc419e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:05.877959 28607 raft_consensus.cc:1261] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [term 1 FOLLOWER]: Refusing update from remote peer aee4cd1825fb42a7a16d5341cacc419e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:05.878849 29169 consensus_queue.cc:1035] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1382721754944169a874a04716552a81" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:05.879166 29163 consensus_queue.cc:1035] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [LEADER]: Connected to new peer: Peer: permanent_uuid: "43217e1ea4f14270a70f12ab0c2a0d49" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 36211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:05.891932 29177 mvcc.cc:205] Tried to move back new op lower bound from 7316866276866691072 to 7316866276647280640. Current Snapshot: MvccSnapshot[applied={T|T < 7316866276866691072}]
I20260810 06:45:05.903550 28353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:05.903798 28353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:05.907425 28353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36fb283b1c1446bcba9295e7a4f14f43: Sending DeleteTablet for 3 replicas of tablet cd1b158641074389bd01bc93e364d663
I20260810 06:45:05.908454 28574 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1b158641074389bd01bc93e364d663 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:05 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:37204
I20260810 06:45:05.908531 29018 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1b158641074389bd01bc93e364d663 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:05 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:59222
I20260810 06:45:05.908919 28804 tablet_service.cc:1514] Processing DeleteTablet for tablet cd1b158641074389bd01bc93e364d663 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:05 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:36992
I20260810 06:45:05.910930 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 28467
Aug 10 06:45:05 dist-test-slave-zj8q krb5kdc[28282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:45:05.935204 29191 tablet_replica.cc:330] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: stopping tablet replica
I20260810 06:45:05.935655 29191 raft_consensus.cc:2229] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:05.938189 29191 raft_consensus.cc:2258] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:05.938817 29189 tablet_replica.cc:330] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: stopping tablet replica
I20260810 06:45:05.939116 29187 tablet_replica.cc:330] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: stopping tablet replica
I20260810 06:45:05.939337 29189 raft_consensus.cc:2229] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:05.939553 29187 raft_consensus.cc:2229] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:05.939965 29189 raft_consensus.cc:2258] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:05.940013 29187 raft_consensus.cc:2258] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:05.940842 29191 ts_tablet_manager.cc:1905] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:05.942180 29189 ts_tablet_manager.cc:1905] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:05.942566 29187 ts_tablet_manager.cc:1905] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:05.948436 29191 ts_tablet_manager.cc:1918] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:05.948633 29191 log.cc:1196] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/cd1b158641074389bd01bc93e364d663
I20260810 06:45:05.949200 29191 ts_tablet_manager.cc:1939] T cd1b158641074389bd01bc93e364d663 P aee4cd1825fb42a7a16d5341cacc419e: Deleting consensus metadata
I20260810 06:45:05.949280 29187 ts_tablet_manager.cc:1918] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:05.949465 29187 log.cc:1196] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/cd1b158641074389bd01bc93e364d663
I20260810 06:45:05.949977 29187 ts_tablet_manager.cc:1939] T cd1b158641074389bd01bc93e364d663 P 43217e1ea4f14270a70f12ab0c2a0d49: Deleting consensus metadata
I20260810 06:45:05.950275 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:05.912028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.10.242.1:36211)
0810 06:45:05.912215 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:05.912222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:05.916131 (+ 3909us) server_negotiation.cc:407] Connection header received
0810 06:45:05.918590 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:05.918596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:05.918704 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:05.918782 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:05.920463 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.921596 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.928380 (+ 6784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:05.929013 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:05.929112 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:05.935064 (+ 5952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:05.935081 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:05.935098 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:05.935157 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:05.939135 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.947033 (+ 7898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.947043 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.947047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.947267 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:05.948094 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:05.948099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:05.948102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:05.948580 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:05.948921 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:05.949565 (+ 644us) server_negotiation.cc:299] Negotiation successful
0810 06:45:05.949814 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:45:05.951049 28333 catalog_manager.cc:4841] TS aee4cd1825fb42a7a16d5341cacc419e (127.10.242.2:45907): tablet cd1b158641074389bd01bc93e364d663 (table test-table [id=42db36c14ced4a3c8cb8abd1ae7f3c11]) successfully deleted
I20260810 06:45:05.951673 28679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:05.951773 28332 catalog_manager.cc:4841] TS 43217e1ea4f14270a70f12ab0c2a0d49 (127.10.242.1:36211): tablet cd1b158641074389bd01bc93e364d663 (table test-table [id=42db36c14ced4a3c8cb8abd1ae7f3c11]) successfully deleted
I20260810 06:45:05.953629 29189 ts_tablet_manager.cc:1918] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:05.953822 29189 log.cc:1196] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/cd1b158641074389bd01bc93e364d663
I20260810 06:45:05.954406 29189 ts_tablet_manager.cc:1939] T cd1b158641074389bd01bc93e364d663 P 1382721754944169a874a04716552a81: Deleting consensus metadata
I20260810 06:45:05.956125 28333 catalog_manager.cc:4841] TS 1382721754944169a874a04716552a81 (127.10.242.3:38009): tablet cd1b158641074389bd01bc93e364d663 (table test-table [id=42db36c14ced4a3c8cb8abd1ae7f3c11]) successfully deleted
I20260810 06:45:06.071412 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 28467
I20260810 06:45:06.101187 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 28699
Aug 10 06:45:06 dist-test-slave-zj8q krb5kdc[28282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:45:06.116731 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:06.102130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.10.242.2:45907)
0810 06:45:06.102354 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:06.102359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:06.102382 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:45:06.102540 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:06.102544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:06.102615 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:06.102703 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:06.103927 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:06.104778 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:06.106079 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:06.106587 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:06.106708 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:06.110939 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:06.110954 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:06.110972 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:06.111009 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:06.113925 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:06.114728 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:06.114735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:06.114738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:06.114884 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:06.115270 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:06.115274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:06.115276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:06.115617 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:06.115968 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:06.116234 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 06:45:06.116399 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 06:45:06.117470 28898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:06.229301 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 28699
I20260810 06:45:06.258965 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 28915
Aug 10 06:45:06 dist-test-slave-zj8q krb5kdc[28282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:45:06.273551 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:06.259710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59232 (local address 127.10.242.3:38009)
0810 06:45:06.259892 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:06.259898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:06.259917 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:45:06.260100 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:06.260104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:06.260185 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:06.260270 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:06.261566 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:06.262350 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:06.263612 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:06.264015 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:06.264125 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:06.268252 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:06.268267 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:06.268285 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:06.268319 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:06.270910 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:06.271564 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:06.271572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:06.271574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:06.271726 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:06.272074 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:06.272078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:06.272080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:06.272448 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:06.272739 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:06.273031 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 06:45:06.273247 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:45:06.274274 29114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:06.360802 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 28915
I20260810 06:45:06.381402 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 28303
I20260810 06:45:06.382603 28412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:06.491717 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 28303
2026-08-10T06:45:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4633 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:06 dist-test-slave-zj8q krb5kdc[29205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:06 dist-test-slave-zj8q krb5kdc[29205](info): set up 2 sockets
Aug 10 06:45:06 dist-test-slave-zj8q krb5kdc[29205](info): commencing operation
krb5kdc: starting...
W20260810 06:45:07.737017 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:45:07 dist-test-slave-zj8q krb5kdc[29205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.242.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.242.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/rangeradmin_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/rangeradmin_127.10.242.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.242.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.242.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/rangerlookup_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/rangerlookup_127.10.242.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.242.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.242.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/HTTP_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/HTTP_127.10.242.20@KRBTEST.COM.keytab.
I20260810 06:45:08.090015 11208 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskcYuaB1/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/postgres -l logfile start
2026-08-10 06:45:10.356 UTC [29257] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:45:10.357 UTC [29257] LOG: listening on IPv4 address "127.10.242.20", port 44837
2026-08-10 06:45:10.360 UTC [29257] LOG: listening on Unix socket "/tmp/.s.PGSQL.44837"
2026-08-10 06:45:10.379 UTC [29261] LOG: database system was shut down at 2026-08-10 06:45:09 UTC
2026-08-10 06:45:10.383 UTC [29257] LOG: database system is ready to accept connections
I20260810 06:45:11.480324 11208 mini_postgres.cc:96] Postgres bound to 44837
2026-08-10 06:45:11.504 UTC [29275] FATAL: database "slave" does not exist
127.10.242.20:44837 - accepting connections
I20260810 06:45:11.505314 11208 mini_ranger.cc:161] Starting Ranger...
I20260810 06:45:11.533733 11208 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:45:11.927734 11208 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:45:11.927861 11208 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-admin
2026-08-10 06:45:12,268 [I] DB FLAVOR :POSTGRES
2026-08-10 06:45:12,269 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:45:12,269 [I] Checking connection..
2026-08-10 06:45:12,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:45:12,528 [I] Checking connection passed.
2026-08-10 06:45:12,528 [I] --------- Verifying version history table ---------
2026-08-10 06:45:12,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:45:12,761 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:45:12,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:45:13,009 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:45:13,009 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:45:13,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:45:13.249 UTC [29386] WARNING: there is no transaction in progress
2026-08-10 06:45:13,257 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:45:13,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:45:13,506 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:45:13,506 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:45:13,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:45:13,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:45:14,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zj8q.c.gcp-upstream.internal','N') ;"
2026-08-10 06:45:14,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:45:14,504 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:45:14,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:45:14,758 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:45:14,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:45:14,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:45:15,235 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:45:15,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:45:16.113 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.128 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.143 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.158 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.173 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.190 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.249 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.255 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.267 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.277 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.286 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.298 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.306 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.311 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.317 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16.323 UTC [29557] WARNING: there is no transaction in progress
2026-08-10 06:45:16,871 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:45:16,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:45:17,119 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:45:17,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:45:17,371 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:45:17,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:45:17,627 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:45:17,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:45:17,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:45:18,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zj8q.c.gcp-upstream.internal';"
2026-08-10 06:45:18,376 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:45:18,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:45:18,608 [I] DB_PATCHES have already been applied
I20260810 06:45:18.613984 11208 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-admin:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskcYuaB1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:45:18.614079 11208 mini_ranger.cc:193] Using host: 127.10.242.20
I20260810 06:45:18.635504 11208 mini_ranger.cc:227] Ranger admin URL: http://127.10.242.20:53879
Aug 10, 2026 6:45:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:45:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:45:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:45:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:45:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:45:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:45:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:45:19.935379 11208 mini_ranger.cc:160] Time spent starting Ranger: real 8.430s user 0.004s sys 0.057s
Aug 10, 2026 6:45:39 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:45:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 6:45:39 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:45:39 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:45:40 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:45:41.107425 11208 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.242.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.242.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/rangerkms_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/rangerkms_127.10.242.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.242.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.242.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.242.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/HTTP_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.242.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/HTTP_127.10.242.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:45:41 dist-test-slave-zj8q krb5kdc[29205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:45:41.293273 11208 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:45:41.316811 11208 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:45:41.993930 11208 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:45:41.994104 11208 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-kms
I20260810 06:45:41.994270 11208 mini_ranger_kms.cc:227] Using postgres at 127.10.242.20:44837
2026-08-10 06:45:42,436 [I] DB FLAVOR :POSTGRES
2026-08-10 06:45:42,436 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:45:42,436 [I] Checking connection
2026-08-10 06:45:42,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:45:42,672 [I] connection success
2026-08-10 06:45:42,672 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:45:42,672 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:45:42,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:45:42,909 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:45:42,909 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:45:42,909 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:45:42,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:44837/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:45:43,154 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:45:43.633471 11208 mini_ranger_kms.cc:310] Created kms service
I20260810 06:45:43.801954 11208 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:45:43.845811 11208 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:45:43 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344341, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:45:44.253336 11208 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:45:44.254570 11208 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-kms:/tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:45:44.254659 11208 mini_ranger_kms.cc:240] Using host: 127.10.242.20
I20260810 06:45:44.280946 11208 mini_ranger_kms.cc:276] Ranger KMS PID: 29851
I20260810 06:45:44.281149 11208 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.242.20:51191
Aug 10, 2026 6:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:45:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 06:45:46 dist-test-slave-zj8q krb5kdc[29205](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344346, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 06:45:46 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344346, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.20@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:45:46.942184 11208 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.649s user 0.118s sys 0.370s
I20260810 06:45:46.942535 11208 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:45:46.942771 11208 mini_ranger_kms.cc:401] 127.10.242.20:51191/kms/v1/keys
Aug 10 06:45:47 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344346, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.20@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
Aug 10 06:45:49 dist-test-slave-zj8q krb5kdc[29205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:45:49.586066 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:39761
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:39761
--ranger_config_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:51191/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:45:49.923951 29938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:49.924224 29938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:49.924350 29938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:49.931272 29938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:45:49.931382 29938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:49.931455 29938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:49.931526 29938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:45:49.931589 29938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:45:49.941429 29938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.242.20:51191/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:39761
--ranger_config_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:39761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:49.942708 29938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:49.943998 29938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:49.957325 29946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:49.957358 29948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:49.957989 29955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:49.958168 29938 server_base.cc:1034] running on GCE node
I20260810 06:45:49.958959 29938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:49.960279 29938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:49.961473 29938 hybrid_clock.cc:648] HybridClock initialized: now 1786344349961425 us; error 41 us; skew 500 ppm
I20260810 06:45:49.961793 29938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:45:49 dist-test-slave-zj8q krb5kdc[29205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344349, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:49.965019 29938 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:45:49.966560 29938 webserver.cc:460] Webserver started at http://127.10.242.62:35755/ using document root <none> and password file <none>
I20260810 06:45:49.967183 29938 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:49.967280 29938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:49.967576 29938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:45:49 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344349, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:45:50.024986 29938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "27453212dd5e4ecaac60e6a98c3b41ee"
format_stamp: "Formatted at 2026-08-10 06:45:50 on dist-test-slave-zj8q"
server_key: "574e30ea5cec77d93dbc2de3fa1a70fb"
server_key_iv: "9ee455d4991cc0d438231737e226a4d7"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:50.025820 29938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "27453212dd5e4ecaac60e6a98c3b41ee"
format_stamp: "Formatted at 2026-08-10 06:45:50 on dist-test-slave-zj8q"
server_key: "574e30ea5cec77d93dbc2de3fa1a70fb"
server_key_iv: "9ee455d4991cc0d438231737e226a4d7"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:50.031255 29938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:45:50.054039 29965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:50.055652 29938 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.005s
I20260810 06:45:50.055840 29938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "27453212dd5e4ecaac60e6a98c3b41ee"
format_stamp: "Formatted at 2026-08-10 06:45:50 on dist-test-slave-zj8q"
server_key: "574e30ea5cec77d93dbc2de3fa1a70fb"
server_key_iv: "9ee455d4991cc0d438231737e226a4d7"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:50.056069 29938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:50.076297 29938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:50.080204 29938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:50.080480 29938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:50.098923 29938 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:39761
I20260810 06:45:50.098927 30044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:39761 every 8 connection(s)
I20260810 06:45:50.100309 29938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:45:50.104048 30046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:50.109335 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 29938
I20260810 06:45:50.109522 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:45:50.110219 11208 external_mini_cluster.cc:1428] Setting key 7d641ac076c65df3179607c9d0305ad1
I20260810 06:45:50.115056 30046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee: Bootstrap starting.
I20260810 06:45:50.118018 30046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:50.119052 30046 log.cc:824] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:50.122015 30046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee: No bootstrap required, opened a new log
Aug 10 06:45:50 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:45:50.131758 30046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27453212dd5e4ecaac60e6a98c3b41ee" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39761 } }
I20260810 06:45:50.132186 30046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:50.132304 30046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27453212dd5e4ecaac60e6a98c3b41ee, State: Initialized, Role: FOLLOWER
I20260810 06:45:50.133039 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:50.112292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.10.242.62:39761)
0810 06:45:50.113130 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:50.113143 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:50.113189 (+ 46us) server_negotiation.cc:407] Connection header received
0810 06:45:50.114884 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:50.114908 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:50.115318 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:50.115667 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:50.116968 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.118373 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.119665 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:50.120370 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:50.120554 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:50.124566 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:50.124577 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:50.124596 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:50.124648 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:50.128541 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:50.129239 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:50.129251 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:50.129260 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:50.129467 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:50.129839 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:50.129844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:50.129848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:50.130617 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:50.130908 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:50.131311 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 06:45:50.131714 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":376,"threads_started":1}
I20260810 06:45:50.133005 30046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27453212dd5e4ecaac60e6a98c3b41ee" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39761 } }
I20260810 06:45:50.133293 30046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:45:50.133438 30046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:45:50.133635 30046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:50.136322 30046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27453212dd5e4ecaac60e6a98c3b41ee" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39761 } }
I20260810 06:45:50.136765 30046 leader_election.cc:302] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27453212dd5e4ecaac60e6a98c3b41ee; no voters:
I20260810 06:45:50.137286 30046 leader_election.cc:288] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:45:50.137459 30054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:50.138505 30054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [term 1 LEADER]: Becoming Leader. State: Replica: 27453212dd5e4ecaac60e6a98c3b41ee, State: Running, Role: LEADER
I20260810 06:45:50.139000 30054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27453212dd5e4ecaac60e6a98c3b41ee" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39761 } }
I20260810 06:45:50.139331 30046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:45:50.141431 30059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27453212dd5e4ecaac60e6a98c3b41ee. Latest consensus state: current_term: 1 leader_uuid: "27453212dd5e4ecaac60e6a98c3b41ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27453212dd5e4ecaac60e6a98c3b41ee" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39761 } } }
I20260810 06:45:50.141746 30059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:50.142285 30055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27453212dd5e4ecaac60e6a98c3b41ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27453212dd5e4ecaac60e6a98c3b41ee" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39761 } } }
I20260810 06:45:50.142575 30055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee [sys.catalog]: This master's current role is: LEADER
I20260810 06:45:50.142994 30066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:45:50.147431 30066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:45:50.152251 30046 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:45:50.165720 30066 catalog_manager.cc:1347] Generated new cluster ID: 62684df0b44045c7a6cfdb6fa1af6128
I20260810 06:45:50.165887 30066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:45:50.180433 30066 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:45:50.182356 30066 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:45:50.200284 30066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee: Generated new TSK 0
I20260810 06:45:50.201143 30066 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:45:50.963248 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:51191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:39761
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:45:51 dist-test-slave-zj8q krb5kdc[29205](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344351, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 06:45:51.385597 30128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:51.386067 30128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:51.386300 30128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:51.397778 30128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:51.397907 30128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:51.398087 30128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:45:51.413394 30128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.242.20:51191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:39761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:51.414932 30128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:51.416368 30128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:51.458509 30147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:51.459172 30146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:51.459218 30149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:51.460450 30128 server_base.cc:1034] running on GCE node
I20260810 06:45:51.461309 30128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:51.462656 30128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:51.463910 30128 hybrid_clock.cc:648] HybridClock initialized: now 1786344351463838 us; error 56 us; skew 500 ppm
I20260810 06:45:51.464285 30128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:45:51 dist-test-slave-zj8q krb5kdc[29205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344351, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:51.468431 30128 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:45:51.470125 30128 webserver.cc:460] Webserver started at http://127.10.242.1:46057/ using document root <none> and password file <none>
I20260810 06:45:51.471016 30128 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:51.471158 30128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:51.471518 30128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:45:51 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344351, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:45:51.490998 30128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "8119423054e341f9a2358590154d72f9"
format_stamp: "Formatted at 2026-08-10 06:45:51 on dist-test-slave-zj8q"
server_key: "1930b4149de98b1b2b2634e2c05974fe"
server_key_iv: "6534a3ce5b2918fbffb83c11293bb30d"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:51.491967 30128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "8119423054e341f9a2358590154d72f9"
format_stamp: "Formatted at 2026-08-10 06:45:51 on dist-test-slave-zj8q"
server_key: "1930b4149de98b1b2b2634e2c05974fe"
server_key_iv: "6534a3ce5b2918fbffb83c11293bb30d"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:51.503762 30128 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.003s sys 0.007s
I20260810 06:45:51.526803 30156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:51.528617 30128 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.002s
I20260810 06:45:51.528885 30128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "8119423054e341f9a2358590154d72f9"
format_stamp: "Formatted at 2026-08-10 06:45:51 on dist-test-slave-zj8q"
server_key: "1930b4149de98b1b2b2634e2c05974fe"
server_key_iv: "6534a3ce5b2918fbffb83c11293bb30d"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:51.529160 30128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:51.556373 30128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:51.561631 30128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:51.562052 30128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:51.566646 30128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:51.568584 30128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:51.568735 30128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:51.568930 30128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:51.569061 30128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:45:51 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344351, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:45:51.781683 30128 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:32905
I20260810 06:45:51.781700 30345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:32905 every 8 connection(s)
I20260810 06:45:51.783128 30128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:45:51.790604 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 30128
I20260810 06:45:51.790833 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:45:51.791620 11208 external_mini_cluster.cc:1428] Setting key 331a9e3eb7c3a131010c1ec8ea735ed4
Aug 10 06:45:51 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344351, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:45:51.804783 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:51.785870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:43313 (local address 127.10.242.62:39761)
0810 06:45:51.786463 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:51.786469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:51.787342 (+ 873us) server_negotiation.cc:407] Connection header received
0810 06:45:51.789273 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:51.789280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:51.789385 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:51.789525 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:51.791107 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.791899 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.794031 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.794500 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.794616 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:51.797869 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:51.797874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:51.797878 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:51.797913 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:51.801039 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:51.801981 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:51.801987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:51.801990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:51.802148 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:51.802729 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:51.802734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:51.802738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:51.803094 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:51.803214 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:51.804208 (+ 994us) server_negotiation.cc:299] Negotiation successful
0810 06:45:51.804434 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":304,"threads_started":1}
I20260810 06:45:51.807224 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:51.786338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:39761 (local address 127.10.242.1:43313)
0810 06:45:51.787152 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:51.787208 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:51.788988 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:51.789731 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:51.789739 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:51.790233 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:51.790904 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:51.790932 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.792067 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.792072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:51.793843 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:51.793868 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:51.794734 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:51.794741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:51.794963 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:51.795603 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:51.795626 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:51.797657 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:51.801218 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:51.801228 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:51.801248 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:51.801846 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:51.802327 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:51.802332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:51.802336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:51.802598 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:51.803328 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:51.803335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:51.803786 (+ 451us) client_negotiation.cc:769] Sending connection context
0810 06:45:51.805281 (+ 1495us) client_negotiation.cc:240] Negotiation successful
0810 06:45:51.805829 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":316,"threads_started":1}
I20260810 06:45:51.809206 30346 heartbeater.cc:347] Connected to a master server at 127.10.242.62:39761
I20260810 06:45:51.809563 30346 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:51.810429 30346 heartbeater.cc:511] Master 127.10.242.62:39761 requested a full tablet report, sending...
I20260810 06:45:51.812919 29994 ts_manager.cc:194] Registered new tserver with Master: 8119423054e341f9a2358590154d72f9 (127.10.242.1:32905)
I20260810 06:45:51.814796 29994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:43313
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:45:51.880847 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:51191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:39761
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:45:52 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344351, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
W20260810 06:45:52.424345 30353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:52.424836 30353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:52.425042 30353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:52.435446 30353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:52.435547 30353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:52.435727 30353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:45:52.445916 30353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.242.20:51191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:39761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:52.447265 30353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:52.448608 30353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:52.463227 30375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:52.463306 30377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:52.463192 30379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:52.464620 30353 server_base.cc:1034] running on GCE node
I20260810 06:45:52.465339 30353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:52.466495 30353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:52.467737 30353 hybrid_clock.cc:648] HybridClock initialized: now 1786344352467685 us; error 34 us; skew 500 ppm
I20260810 06:45:52.468112 30353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:45:52 dist-test-slave-zj8q krb5kdc[29205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344352, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:52.472194 30353 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:45:52.473863 30353 webserver.cc:460] Webserver started at http://127.10.242.2:42723/ using document root <none> and password file <none>
I20260810 06:45:52.474762 30353 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:52.474912 30353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:52.475270 30353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:45:52 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344352, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:45:52.494767 30353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "a727ab51d1d840c6a663e12bf647af6d"
format_stamp: "Formatted at 2026-08-10 06:45:52 on dist-test-slave-zj8q"
server_key: "f7825479b890f10df8ca5da4cad1ef9b"
server_key_iv: "fd7df295e8639fe2c05ea488e293f6f8"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:52.495859 30353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "a727ab51d1d840c6a663e12bf647af6d"
format_stamp: "Formatted at 2026-08-10 06:45:52 on dist-test-slave-zj8q"
server_key: "f7825479b890f10df8ca5da4cad1ef9b"
server_key_iv: "fd7df295e8639fe2c05ea488e293f6f8"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:52.501543 30353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:45:52.530930 30390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:52.533159 30353 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.009s sys 0.003s
I20260810 06:45:52.533370 30353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "a727ab51d1d840c6a663e12bf647af6d"
format_stamp: "Formatted at 2026-08-10 06:45:52 on dist-test-slave-zj8q"
server_key: "f7825479b890f10df8ca5da4cad1ef9b"
server_key_iv: "fd7df295e8639fe2c05ea488e293f6f8"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:52.533646 30353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:52.557405 30353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:52.562556 30353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:52.562901 30353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:52.564137 30353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:52.565974 30353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:52.566082 30353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:52.566277 30353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:52.566344 30353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:52.620563 30353 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:33299
I20260810 06:45:52.620627 30560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:33299 every 8 connection(s)
I20260810 06:45:52.622242 30353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:45:52.626601 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 30353
I20260810 06:45:52.626791 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:45:52.627365 11208 external_mini_cluster.cc:1428] Setting key dda87e5392badb27d2e0778ee0fbc5b1
Aug 10 06:45:52 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344352, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:45:52.656353 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:52.625193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:52571 (local address 127.10.242.62:39761)
0810 06:45:52.635887 (+ 10694us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:52.635893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:52.635918 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:45:52.638241 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:52.638247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:52.638328 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:52.638403 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:52.640577 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.641571 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.643928 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.644506 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.644646 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:52.649194 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:52.649200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:52.649204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:52.649254 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:52.652606 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:52.653491 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:52.653497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:52.653501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:52.653685 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:52.654264 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:52.654269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:52.654272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:52.654624 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:52.654757 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:52.655757 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0810 06:45:52.655985 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10550,"thread_start_us":10423,"threads_started":1}
I20260810 06:45:52.658813 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:52.625829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:39761 (local address 127.10.242.2:52571)
0810 06:45:52.634877 (+ 9048us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:52.634949 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:52.637684 (+ 2735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:52.638948 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:52.638960 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:52.639564 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:52.640384 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:52.640409 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.641862 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.641867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:52.643767 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:52.643782 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:52.645000 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:52.645007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:52.645306 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:52.646125 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:52.646181 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:52.648921 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:52.652782 (+ 3861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:52.652792 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:52.652811 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:52.653361 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:52.653852 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:52.653856 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:52.653860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:52.654120 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:52.654849 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:52.654857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:52.655308 (+ 451us) client_negotiation.cc:769] Sending connection context
0810 06:45:52.656823 (+ 1515us) client_negotiation.cc:240] Negotiation successful
0810 06:45:52.657397 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8666,"thread_start_us":8446,"threads_started":1}
I20260810 06:45:52.660701 30562 heartbeater.cc:347] Connected to a master server at 127.10.242.62:39761
I20260810 06:45:52.661029 30562 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:52.661888 30562 heartbeater.cc:511] Master 127.10.242.62:39761 requested a full tablet report, sending...
I20260810 06:45:52.663872 29994 ts_manager.cc:194] Registered new tserver with Master: a727ab51d1d840c6a663e12bf647af6d (127.10.242.2:33299)
I20260810 06:45:52.664672 29994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:52571
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:45:52.702720 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:51191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:39761
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:45:52.818332 30346 heartbeater.cc:503] Master 127.10.242.62:39761 was elected leader, sending a full tablet report...
W20260810 06:45:53.043195 30570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:45:53.043470 30570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:45:53.043613 30570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:45:53.050721 30570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:45:53.050830 30570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:45:53.050974 30570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:45:53.060928 30570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.242.20:51191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:39761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:45:53.062335 30570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:45:53.063632 30570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:45:53.076649 30582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:53.076642 30581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:45:53.076654 30584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:45:53.082619 30570 server_base.cc:1034] running on GCE node
I20260810 06:45:53.083258 30570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:45:53.084259 30570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:45:53.085477 30570 hybrid_clock.cc:648] HybridClock initialized: now 1786344353085423 us; error 44 us; skew 500 ppm
I20260810 06:45:53.085832 30570 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:45:53 dist-test-slave-zj8q krb5kdc[29205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344353, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:45:53.089192 30570 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:45:53.090689 30570 webserver.cc:460] Webserver started at http://127.10.242.3:33469/ using document root <none> and password file <none>
I20260810 06:45:53.091364 30570 fs_manager.cc:359] Metadata directory not provided
I20260810 06:45:53.091476 30570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:45:53.091782 30570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:45:53 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344353, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:45:53.108108 30570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "74a7f164447f47f49ef6a358661f47f6"
format_stamp: "Formatted at 2026-08-10 06:45:53 on dist-test-slave-zj8q"
server_key: "5b0809054fa004a50808b2e47eca09dd"
server_key_iv: "c7dd6d6b38e8b6dd7d919f93138b75b1"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:53.109196 30570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "74a7f164447f47f49ef6a358661f47f6"
format_stamp: "Formatted at 2026-08-10 06:45:53 on dist-test-slave-zj8q"
server_key: "5b0809054fa004a50808b2e47eca09dd"
server_key_iv: "c7dd6d6b38e8b6dd7d919f93138b75b1"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:53.116329 30570 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260810 06:45:53.131665 30598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:53.133406 30570 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:45:53.133651 30570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "74a7f164447f47f49ef6a358661f47f6"
format_stamp: "Formatted at 2026-08-10 06:45:53 on dist-test-slave-zj8q"
server_key: "5b0809054fa004a50808b2e47eca09dd"
server_key_iv: "c7dd6d6b38e8b6dd7d919f93138b75b1"
server_key_version: "kuduclusterkey@0"
I20260810 06:45:53.133961 30570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:45:53.153805 30570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:45:53.157512 30570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:45:53.157802 30570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:45:53.159077 30570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:45:53.160625 30570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:45:53.160702 30570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:53.160827 30570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:45:53.160881 30570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:45:53.191789 30570 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:38765
I20260810 06:45:53.191833 30761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:38765 every 8 connection(s)
I20260810 06:45:53.193025 30570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:45:53.199290 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 30570
I20260810 06:45:53.199429 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:45:53.199855 11208 external_mini_cluster.cc:1428] Setting key 7122232f658a2e8f222298ce54e023f7
Aug 10 06:45:53 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344353, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:45:53.211942 30767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.195465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:42955 (local address 127.10.242.62:39761)
0810 06:45:53.195969 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.195974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.196732 (+ 758us) server_negotiation.cc:407] Connection header received
0810 06:45:53.198589 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.198596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.198689 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.198782 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:53.200286 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.201192 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.203067 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.203484 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.203592 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.206658 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:53.206662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:53.206666 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:53.206707 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:53.209067 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.209739 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.209743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.209745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.209871 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.210325 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.210329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.210331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.210596 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:53.210757 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.211370 (+ 613us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.211582 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":277,"threads_started":1}
I20260810 06:45:53.212775 30768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.195850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:39761 (local address 127.10.242.3:42955)
0810 06:45:53.196557 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:53.196600 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:53.198310 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:53.199026 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:53.199034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:53.199501 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:45:53.200106 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.200126 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.201341 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.201345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.202919 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.202939 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.203694 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.203702 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.203911 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.204461 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:45:53.204482 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:45:53.206453 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:45:53.209200 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:53.209209 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:53.209224 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:53.209616 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:53.209971 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:45:53.209976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:45:53.209980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:45:53.210211 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:45:53.210720 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:45:53.210727 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:45:53.211030 (+ 303us) client_negotiation.cc:769] Sending connection context
0810 06:45:53.211253 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 06:45:53.211713 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":288,"threads_started":1}
I20260810 06:45:53.214179 30763 heartbeater.cc:347] Connected to a master server at 127.10.242.62:39761
I20260810 06:45:53.214517 30763 heartbeater.cc:464] Registering TS with master...
I20260810 06:45:53.215349 30763 heartbeater.cc:511] Master 127.10.242.62:39761 requested a full tablet report, sending...
I20260810 06:45:53.216746 29993 ts_manager.cc:194] Registered new tserver with Master: 74a7f164447f47f49ef6a358661f47f6 (127.10.242.3:38765)
I20260810 06:45:53.217675 29993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:42955
I20260810 06:45:53.227138 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:45:53.244387 30767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.231685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54456 (local address 127.10.242.62:39761)
0810 06:45:53.231855 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.231863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.232254 (+ 391us) server_negotiation.cc:407] Connection header received
0810 06:45:53.232413 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.232419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.232500 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.232578 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:53.234096 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.234946 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.236270 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.236695 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.236807 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.239321 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:53.239325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:53.239329 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:53.239360 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:53.242094 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.242751 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.242755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.242758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.242885 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.243159 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.243161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.243164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.243432 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:53.243599 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.243903 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.244066 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:45:53.248361 29993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:45:53.255303 29993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:45:53.279186 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.267456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.10.242.3:38765)
0810 06:45:53.270432 (+ 2976us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.270440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.270474 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:45:53.270594 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.270606 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.270869 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.271782 (+ 913us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:53.272062 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.273707 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.276585 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.278001 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.278133 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.278397 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.278667 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.278828 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2781,"thread_start_us":2602,"threads_started":1}
I20260810 06:45:53.279245 30783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.267218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38765 (local address 127.0.0.1:45024)
0810 06:45:53.268798 (+ 1580us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:53.268852 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:53.269025 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:53.271113 (+ 2088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:53.271122 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:53.271153 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:53.271567 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.271579 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.273858 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.273864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.276431 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.276447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.278214 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.278221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.278423 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.278481 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 06:45:53.278626 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 06:45:53.278860 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1309,"mutex_wait_us":27,"thread_start_us":559,"threads_started":1}
I20260810 06:45:53.281504 30790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.268201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.10.242.2:33299)
0810 06:45:53.274940 (+ 6739us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.274948 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.274977 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:45:53.275072 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.275079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.275314 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.275491 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:53.276133 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.277234 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.279616 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.280492 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.280614 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.280802 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.281058 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.281226 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6578,"thread_start_us":6466,"threads_started":1}
I20260810 06:45:53.281880 30789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.267611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:33299 (local address 127.0.0.1:58848)
0810 06:45:53.269701 (+ 2090us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:53.269718 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:53.269823 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:53.275492 (+ 5669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:53.275497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:53.275520 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:53.275919 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.275930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.277380 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.277385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.279452 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.279466 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.280725 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.280732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.280901 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.280924 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:45:53.280999 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:45:53.281172 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1970,"mutex_wait_us":43,"thread_start_us":1819,"threads_started":1}
I20260810 06:45:53.283147 30669 tablet_service.cc:1467] Processing CreateTablet for tablet 09dae42969c94cbab2942312d129e43d (DEFAULT_TABLE table=test-table [id=7fe4ae5034db4889b219e300ce3aa4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:53.284400 30791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.269821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:32905 (local address 127.0.0.1:49758)
0810 06:45:53.275135 (+ 5314us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:53.275152 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:53.275291 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:53.275894 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:53.275901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:53.275925 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:53.276283 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.276291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.278245 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.278251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.281807 (+ 3556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.281827 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.283633 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.283639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.283805 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.283837 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:45:53.283924 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:45:53.284039 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5187,"mutex_wait_us":3,"thread_start_us":5035,"threads_started":1}
I20260810 06:45:53.284626 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.270461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49758 (local address 127.10.242.1:32905)
0810 06:45:53.274501 (+ 4040us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.274510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.275412 (+ 902us) server_negotiation.cc:407] Connection header received
0810 06:45:53.275507 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.275514 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.275702 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.275881 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:53.276454 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.278046 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.282002 (+ 3956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.283393 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.283543 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.283758 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.284008 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.284165 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3871,"thread_start_us":3731,"threads_started":1}
I20260810 06:45:53.284782 30669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09dae42969c94cbab2942312d129e43d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:53.285199 30462 tablet_service.cc:1467] Processing CreateTablet for tablet 09dae42969c94cbab2942312d129e43d (DEFAULT_TABLE table=test-table [id=7fe4ae5034db4889b219e300ce3aa4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:53.286379 30462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09dae42969c94cbab2942312d129e43d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:53.288142 30233 tablet_service.cc:1467] Processing CreateTablet for tablet 09dae42969c94cbab2942312d129e43d (DEFAULT_TABLE table=test-table [id=7fe4ae5034db4889b219e300ce3aa4ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:45:53.289552 30233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09dae42969c94cbab2942312d129e43d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:45:53.297122 30800 tablet_bootstrap.cc:492] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: Bootstrap starting.
I20260810 06:45:53.300022 30801 tablet_bootstrap.cc:492] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: Bootstrap starting.
I20260810 06:45:53.301194 30800 tablet_bootstrap.cc:654] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:53.302616 30800 log.cc:824] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:53.302677 30803 tablet_bootstrap.cc:492] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: Bootstrap starting.
I20260810 06:45:53.303411 30801 tablet_bootstrap.cc:654] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:53.304450 30801 log.cc:824] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:53.305950 30800 tablet_bootstrap.cc:492] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: No bootstrap required, opened a new log
I20260810 06:45:53.306094 30803 tablet_bootstrap.cc:654] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: Neither blocks nor log segments found. Creating new log.
I20260810 06:45:53.306326 30800 ts_tablet_manager.cc:1397] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260810 06:45:53.307336 30803 log.cc:824] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:45:53.307529 30801 tablet_bootstrap.cc:492] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: No bootstrap required, opened a new log
I20260810 06:45:53.307822 30801 ts_tablet_manager.cc:1397] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:45:53.309648 30803 tablet_bootstrap.cc:492] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: No bootstrap required, opened a new log
I20260810 06:45:53.309880 30803 ts_tablet_manager.cc:1397] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260810 06:45:53.316514 30800 raft_consensus.cc:348] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } }
I20260810 06:45:53.316948 30800 raft_consensus.cc:374] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:53.316782 30803 raft_consensus.cc:348] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } }
I20260810 06:45:53.317085 30800 raft_consensus.cc:729] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a727ab51d1d840c6a663e12bf647af6d, State: Initialized, Role: FOLLOWER
I20260810 06:45:53.317163 30803 raft_consensus.cc:374] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:53.317270 30803 raft_consensus.cc:729] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8119423054e341f9a2358590154d72f9, State: Initialized, Role: FOLLOWER
I20260810 06:45:53.317795 30803 consensus_queue.cc:260] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } }
I20260810 06:45:53.317808 30800 consensus_queue.cc:260] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } }
I20260810 06:45:53.317817 30801 raft_consensus.cc:348] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } }
I20260810 06:45:53.318200 30801 raft_consensus.cc:374] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:45:53.318360 30801 raft_consensus.cc:729] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74a7f164447f47f49ef6a358661f47f6, State: Initialized, Role: FOLLOWER
I20260810 06:45:53.319110 30801 consensus_queue.cc:260] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } }
I20260810 06:45:53.319442 30803 ts_tablet_manager.cc:1428] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: Time spent starting tablet: real 0.009s user 0.000s sys 0.010s
I20260810 06:45:53.322813 30562 heartbeater.cc:503] Master 127.10.242.62:39761 was elected leader, sending a full tablet report...
I20260810 06:45:53.323239 30807 raft_consensus.cc:482] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:45:53.323552 30807 raft_consensus.cc:504] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } }
I20260810 06:45:53.324911 30800 ts_tablet_manager.cc:1428] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: Time spent starting tablet: real 0.018s user 0.012s sys 0.004s
I20260810 06:45:53.328572 30807 leader_election.cc:288] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74a7f164447f47f49ef6a358661f47f6 (127.10.242.3:38765), a727ab51d1d840c6a663e12bf647af6d (127.10.242.2:33299)
I20260810 06:45:53.328649 30763 heartbeater.cc:503] Master 127.10.242.62:39761 was elected leader, sending a full tablet report...
I20260810 06:45:53.329253 30801 ts_tablet_manager.cc:1428] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: Time spent starting tablet: real 0.021s user 0.008s sys 0.012s
I20260810 06:45:53.335777 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.326565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:33299 (local address 127.10.242.1:53323)
0810 06:45:53.329226 (+ 2661us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:53.329247 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:53.329386 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:53.330314 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:53.330320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:53.330343 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:53.330713 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.330725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.331849 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.331854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.333629 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.333642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.334991 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.334998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.335197 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.335233 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:45:53.335330 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:45:53.335429 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2524,"thread_start_us":3575,"threads_started":1}
I20260810 06:45:53.335847 30790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.326900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:53323 (local address 127.10.242.2:33299)
0810 06:45:53.327090 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.327097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.329922 (+ 2825us) server_negotiation.cc:407] Connection header received
0810 06:45:53.329997 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.330002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.330085 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.330202 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:53.330891 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.331680 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.333789 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.334777 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.334927 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.334988 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.335452 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.335582 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:45:53.335938 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.327314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:37437 (local address 127.10.242.3:38765)
0810 06:45:53.327513 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.327520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.329994 (+ 2474us) server_negotiation.cc:407] Connection header received
0810 06:45:53.330179 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.330184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.330256 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.330340 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:45:53.331107 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.332093 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.333986 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.335031 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.335191 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.335257 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.335590 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.335722 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:45:53.336234 30812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.326709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:38765 (local address 127.10.242.1:37437)
0810 06:45:53.329917 (+ 3208us) negotiation.cc:107] Waiting for socket to connect
0810 06:45:53.329935 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:45:53.330044 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:45:53.330493 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:45:53.330497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:45:53.330520 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:45:53.330932 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.330945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.332220 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.332225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.333828 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:45:53.333846 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.335232 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.335236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:45:53.335426 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.335458 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:45:53.335561 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 06:45:53.335677 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3074,"thread_start_us":2888,"threads_started":1}
I20260810 06:45:53.336884 30500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09dae42969c94cbab2942312d129e43d" candidate_uuid: "8119423054e341f9a2358590154d72f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727ab51d1d840c6a663e12bf647af6d" is_pre_election: true
I20260810 06:45:53.337143 30699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09dae42969c94cbab2942312d129e43d" candidate_uuid: "8119423054e341f9a2358590154d72f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74a7f164447f47f49ef6a358661f47f6" is_pre_election: true
I20260810 06:45:53.337308 30500 raft_consensus.cc:2454] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8119423054e341f9a2358590154d72f9 in term 0.
I20260810 06:45:53.337561 30699 raft_consensus.cc:2454] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8119423054e341f9a2358590154d72f9 in term 0.
I20260810 06:45:53.337990 30159 leader_election.cc:302] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8119423054e341f9a2358590154d72f9, a727ab51d1d840c6a663e12bf647af6d; no voters:
I20260810 06:45:53.338788 30807 raft_consensus.cc:2781] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:45:53.338955 30807 raft_consensus.cc:482] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:45:53.339075 30807 raft_consensus.cc:3037] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:53.341424 30807 raft_consensus.cc:504] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } }
I20260810 06:45:53.342034 30807 leader_election.cc:288] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [CANDIDATE]: Term 1 election: Requested vote from peers 74a7f164447f47f49ef6a358661f47f6 (127.10.242.3:38765), a727ab51d1d840c6a663e12bf647af6d (127.10.242.2:33299)
I20260810 06:45:53.342376 30699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09dae42969c94cbab2942312d129e43d" candidate_uuid: "8119423054e341f9a2358590154d72f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74a7f164447f47f49ef6a358661f47f6"
I20260810 06:45:53.342536 30699 raft_consensus.cc:3037] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:53.342501 30500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09dae42969c94cbab2942312d129e43d" candidate_uuid: "8119423054e341f9a2358590154d72f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727ab51d1d840c6a663e12bf647af6d"
I20260810 06:45:53.342690 30500 raft_consensus.cc:3037] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:45:53.344635 30699 raft_consensus.cc:2454] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8119423054e341f9a2358590154d72f9 in term 1.
I20260810 06:45:53.344851 30500 raft_consensus.cc:2454] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8119423054e341f9a2358590154d72f9 in term 1.
I20260810 06:45:53.344977 30163 leader_election.cc:302] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74a7f164447f47f49ef6a358661f47f6, 8119423054e341f9a2358590154d72f9; no voters:
I20260810 06:45:53.345268 30807 raft_consensus.cc:2781] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:45:53.345696 30807 raft_consensus.cc:686] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 1 LEADER]: Becoming Leader. State: Replica: 8119423054e341f9a2358590154d72f9, State: Running, Role: LEADER
I20260810 06:45:53.346112 30807 consensus_queue.cc:237] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } }
I20260810 06:45:53.350579 29993 catalog_manager.cc:5495] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8119423054e341f9a2358590154d72f9 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "8119423054e341f9a2358590154d72f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8119423054e341f9a2358590154d72f9" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 32905 } health_report { overall_health: HEALTHY } } }
I20260810 06:45:53.365818 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.360711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49770 (local address 127.10.242.1:32905)
0810 06:45:53.360901 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.360907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.360922 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:45:53.360989 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.360992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.361075 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.361204 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:45:53.361595 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.362360 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.363811 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.364211 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.364315 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.364802 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:45:53.365256 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:45:53.365383 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.365486 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.365582 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:45:53.373142 30500 raft_consensus.cc:1261] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [term 1 FOLLOWER]: Refusing update from remote peer 8119423054e341f9a2358590154d72f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:53.373214 30699 raft_consensus.cc:1261] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [term 1 FOLLOWER]: Refusing update from remote peer 8119423054e341f9a2358590154d72f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:45:53.374011 30813 consensus_queue.cc:1035] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a727ab51d1d840c6a663e12bf647af6d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 33299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:53.374382 30807 consensus_queue.cc:1035] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74a7f164447f47f49ef6a358661f47f6" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 38765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:45:53.384860 30818 mvcc.cc:205] Tried to move back new op lower bound from 7316866471407927296 to 7316866471309373440. Current Snapshot: MvccSnapshot[applied={T|T < 7316866471407927296}]
I20260810 06:45:53.386370 30820 mvcc.cc:205] Tried to move back new op lower bound from 7316866471407927296 to 7316866471309373440. Current Snapshot: MvccSnapshot[applied={T|T < 7316866471407927296}]
I20260810 06:45:53.386448 30821 mvcc.cc:205] Tried to move back new op lower bound from 7316866471407927296 to 7316866471309373440. Current Snapshot: MvccSnapshot[applied={T|T < 7316866471407927296}]
I20260810 06:45:53.397565 29994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:53.397822 29994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:45:53.401618 29994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27453212dd5e4ecaac60e6a98c3b41ee: Sending DeleteTablet for 3 replicas of tablet 09dae42969c94cbab2942312d129e43d
I20260810 06:45:53.402661 30669 tablet_service.cc:1514] Processing DeleteTablet for tablet 09dae42969c94cbab2942312d129e43d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:53 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:45024
I20260810 06:45:53.403095 30462 tablet_service.cc:1514] Processing DeleteTablet for tablet 09dae42969c94cbab2942312d129e43d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:53 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:58848
I20260810 06:45:53.403430 30233 tablet_service.cc:1514] Processing DeleteTablet for tablet 09dae42969c94cbab2942312d129e43d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:45:53 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:49758
I20260810 06:45:53.405372 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 30128
I20260810 06:45:53.415827 30831 tablet_replica.cc:330] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: stopping tablet replica
I20260810 06:45:53.416329 30831 raft_consensus.cc:2229] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:53.416891 30831 raft_consensus.cc:2258] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:53.417636 30832 tablet_replica.cc:330] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: stopping tablet replica
I20260810 06:45:53.417968 30832 raft_consensus.cc:2229] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:45:53.418675 30833 tablet_replica.cc:330] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: stopping tablet replica
I20260810 06:45:53.419123 30833 raft_consensus.cc:2229] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:45:53.419844 30833 raft_consensus.cc:2258] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:53.420205 30831 ts_tablet_manager.cc:1905] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:53.422130 30833 ts_tablet_manager.cc:1905] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:45:53 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:45:53.427155 30831 ts_tablet_manager.cc:1918] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:53.427388 30831 log.cc:1196] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/09dae42969c94cbab2942312d129e43d
I20260810 06:45:53.427974 30831 ts_tablet_manager.cc:1939] T 09dae42969c94cbab2942312d129e43d P 74a7f164447f47f49ef6a358661f47f6: Deleting consensus metadata
I20260810 06:45:53.429679 30833 ts_tablet_manager.cc:1918] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:53.429895 30833 log.cc:1196] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/09dae42969c94cbab2942312d129e43d
I20260810 06:45:53.429916 29975 catalog_manager.cc:4841] TS 74a7f164447f47f49ef6a358661f47f6 (127.10.242.3:38765): tablet 09dae42969c94cbab2942312d129e43d (table test-table [id=7fe4ae5034db4889b219e300ce3aa4ba]) successfully deleted
I20260810 06:45:53.430290 30832 raft_consensus.cc:2258] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:45:53.430483 30833 ts_tablet_manager.cc:1939] T 09dae42969c94cbab2942312d129e43d P 8119423054e341f9a2358590154d72f9: Deleting consensus metadata
I20260810 06:45:53.432194 30832 ts_tablet_manager.cc:1905] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:45:53.432718 29966 catalog_manager.cc:4841] TS 8119423054e341f9a2358590154d72f9 (127.10.242.1:32905): tablet 09dae42969c94cbab2942312d129e43d (table test-table [id=7fe4ae5034db4889b219e300ce3aa4ba]) successfully deleted
I20260810 06:45:53.434084 30792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.406858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.10.242.1:32905)
0810 06:45:53.407036 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.407042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.411268 (+ 4226us) server_negotiation.cc:407] Connection header received
0810 06:45:53.412073 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.412079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.412165 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.412237 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:53.415298 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.416556 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.418857 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.419567 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.419682 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.425385 (+ 5703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:53.425399 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:53.425417 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:53.425460 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:53.430043 (+ 4583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.431297 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.431306 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.431309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.431520 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.432053 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.432060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.432063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.432536 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:53.432829 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.433428 (+ 599us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.433672 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:45:53.434803 30336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:53.438539 30832 ts_tablet_manager.cc:1918] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:45:53.438690 30832 log.cc:1196] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/09dae42969c94cbab2942312d129e43d
I20260810 06:45:53.439239 30832 ts_tablet_manager.cc:1939] T 09dae42969c94cbab2942312d129e43d P a727ab51d1d840c6a663e12bf647af6d: Deleting consensus metadata
I20260810 06:45:53.440770 29966 catalog_manager.cc:4841] TS a727ab51d1d840c6a663e12bf647af6d (127.10.242.2:33299): tablet 09dae42969c94cbab2942312d129e43d (table test-table [id=7fe4ae5034db4889b219e300ce3aa4ba]) successfully deleted
I20260810 06:45:53.531476 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 30128
I20260810 06:45:53.554179 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 30353
Aug 10 06:45:53 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:45:53.569016 30790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.554994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.10.242.2:33299)
0810 06:45:53.555196 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.555203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.555224 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:45:53.555369 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.555373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.555439 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.555521 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:53.556807 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.557626 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.559054 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.559682 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.559800 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.563958 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:53.563971 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:53.563987 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:53.564025 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:53.566684 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.567281 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.567289 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.567291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.567424 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.567733 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.567736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.567738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.568103 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:53.568336 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.568589 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.568757 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:45:53.569749 30553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:53.658789 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 30353
I20260810 06:45:53.686120 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 30570
Aug 10 06:45:53 dist-test-slave-zj8q krb5kdc[29205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:45:53.700420 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:45:53.686983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.10.242.3:38765)
0810 06:45:53.687147 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:45:53.687168 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 06:45:53.687187 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:45:53.687319 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:45:53.687322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:45:53.687387 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:45:53.687469 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:45:53.688783 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.689517 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.690735 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:45:53.691102 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:45:53.691199 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:45:53.695326 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:45:53.695343 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:45:53.695362 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:45:53.695401 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:45:53.698025 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.698693 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.698701 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.698703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.698848 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:45:53.699135 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:45:53.699138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:45:53.699139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:45:53.699500 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:45:53.699718 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 06:45:53.699976 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 06:45:53.700138 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:45:53.701027 30750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:53.791066 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 30570
I20260810 06:45:53.817062 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 29938
I20260810 06:45:53.818245 30037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:45:53.928844 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 29938
I20260810 06:45:53.958299 11208 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:45:54.302654 11208 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:45:54.302948 11208 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:45:54.647054 11208 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:45:54.647 UTC [29257] LOG: received smart shutdown request
2026-08-10 06:45:54.651 UTC [29257] LOG: background worker "logical replication launcher" (PID 29270) exited with exit code 1
2026-08-10 06:45:54.651 UTC [29264] LOG: shutting down
2026-08-10 06:45:54.777 UTC [29257] LOG: database system is shut down
2026-08-10T06:45:54Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48385 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:45:54 dist-test-slave-zj8q krb5kdc[30856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:45:54 dist-test-slave-zj8q krb5kdc[30856](info): set up 2 sockets
Aug 10 06:45:54 dist-test-slave-zj8q krb5kdc[30856](info): commencing operation
krb5kdc: starting...
W20260810 06:45:56.129218 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:45:56 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:45:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:45:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.242.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.242.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/rangeradmin_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/rangeradmin_127.10.242.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.242.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.242.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/rangerlookup_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/rangerlookup_127.10.242.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.242.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.242.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/HTTP_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/HTTP_127.10.242.20@KRBTEST.COM.keytab.
I20260810 06:45:56.478405 11208 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskcYuaB1/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/postgres -l logfile start
2026-08-10 06:45:58.653 UTC [30907] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:45:58.653 UTC [30907] LOG: listening on IPv4 address "127.10.242.20", port 39691
2026-08-10 06:45:58.656 UTC [30907] LOG: listening on Unix socket "/tmp/.s.PGSQL.39691"
2026-08-10 06:45:58.677 UTC [30913] LOG: database system was shut down at 2026-08-10 06:45:57 UTC
2026-08-10 06:45:58.680 UTC [30907] LOG: database system is ready to accept connections
I20260810 06:45:59.789534 11208 mini_postgres.cc:96] Postgres bound to 39691
2026-08-10 06:45:59.811 UTC [30925] FATAL: database "slave" does not exist
127.10.242.20:39691 - accepting connections
I20260810 06:45:59.811946 11208 mini_ranger.cc:161] Starting Ranger...
I20260810 06:45:59.838694 11208 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:46:00.224588 11208 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:46:00.224733 11208 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-admin
2026-08-10 06:46:00,515 [I] DB FLAVOR :POSTGRES
2026-08-10 06:46:00,516 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:46:00,516 [I] Checking connection..
2026-08-10 06:46:00,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:46:00,774 [I] Checking connection passed.
2026-08-10 06:46:00,775 [I] --------- Verifying version history table ---------
2026-08-10 06:46:00,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:46:01,033 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:46:01,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:46:01,293 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:46:01,293 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:46:01,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:46:01.516 UTC [31022] WARNING: there is no transaction in progress
2026-08-10 06:46:01,525 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:46:01,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:46:01,760 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:46:01,761 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:46:01,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:46:02,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:46:02,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zj8q.c.gcp-upstream.internal','N') ;"
2026-08-10 06:46:02,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:46:02,744 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:46:02,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:46:02,981 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:46:02,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:46:03,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:46:03,474 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:46:03,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:46:04.365 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.381 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.396 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.410 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.425 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.441 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.490 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.496 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.508 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.516 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.525 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.537 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.545 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.551 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.557 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:04.563 UTC [31192] WARNING: there is no transaction in progress
2026-08-10 06:46:05,107 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:46:05,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:46:05,356 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:46:05,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:46:05,615 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:46:05,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:46:05,871 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:46:05,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:46:06,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:46:06,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zj8q.c.gcp-upstream.internal';"
2026-08-10 06:46:06,601 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:46:06,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:46:06,859 [I] DB_PATCHES have already been applied
I20260810 06:46:06.865181 11208 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-admin:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskcYuaB1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:46:06.865271 11208 mini_ranger.cc:193] Using host: 127.10.242.20
I20260810 06:46:06.884788 11208 mini_ranger.cc:227] Ranger admin URL: http://127.10.242.20:35095
Aug 10, 2026 6:46:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:46:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:46:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:46:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:46:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:46:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:46:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:46:08.185016 11208 mini_ranger.cc:160] Time spent starting Ranger: real 8.373s user 0.003s sys 0.050s
Aug 10, 2026 6:46:26 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:46:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 6:46:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:46:26 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:46:27 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:46:28.321565 11208 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.242.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.242.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/rangerkms_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.242.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/rangerkms_127.10.242.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.242.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.242.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.242.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/HTTP_127.10.242.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.242.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/HTTP_127.10.242.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:46:28 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344388, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:46:28.506346 11208 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:46:28.529458 11208 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:46:29.207275 11208 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:46:29.207369 11208 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-kms
I20260810 06:46:29.207401 11208 mini_ranger_kms.cc:227] Using postgres at 127.10.242.20:39691
2026-08-10 06:46:29,552 [I] DB FLAVOR :POSTGRES
2026-08-10 06:46:29,552 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:46:29,552 [I] Checking connection
2026-08-10 06:46:29,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:46:29,786 [I] connection success
2026-08-10 06:46:29,787 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:46:29,787 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:46:29,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:46:30,036 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:46:30,036 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:46:30,036 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:46:30,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.242.20:39691/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:46:30,286 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:46:30.735697 11208 mini_ranger_kms.cc:310] Created kms service
I20260810 06:46:30.912166 11208 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:46:30.954819 11208 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:46:30 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344388, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:31.345623 11208 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:46:31.345961 11208 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-kms:/tmp/dist-test-taskcYuaB1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskcYuaB1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:46:31.346029 11208 mini_ranger_kms.cc:240] Using host: 127.10.242.20
I20260810 06:46:31.367735 11208 mini_ranger_kms.cc:276] Ranger KMS PID: 31473
I20260810 06:46:31.367868 11208 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.242.20:46263
Aug 10, 2026 6:46:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:46:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:46:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:46:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:46:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:46:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 06:46:33 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344393, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:33.867813 11208 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.361s user 0.063s sys 0.312s
I20260810 06:46:33.868046 11208 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:46:33.868204 11208 mini_ranger_kms.cc:401] 127.10.242.20:46263/kms/v1/keys
Aug 10 06:46:34 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344393, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.20@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
Aug 10 06:46:34 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344393, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.242.20@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
Aug 10 06:46:36 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:46:36.666181 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:42711
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:42711
--ranger_config_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:46263/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:36.995653 31567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:36.995927 31567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:36.996059 31567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:37.002970 31567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:37.003082 31567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:37.003160 31567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:37.003239 31567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:37.003304 31567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:37.012961 31567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.242.20:46263/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:42711
--ranger_config_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:42711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:37.014209 31567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:37.015525 31567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:37.027045 31576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:37.027009 31584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:37.026974 31579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:37.027890 31567 server_base.cc:1034] running on GCE node
I20260810 06:46:37.028604 31567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:37.029875 31567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:37.031062 31567 hybrid_clock.cc:648] HybridClock initialized: now 1786344397031027 us; error 27 us; skew 500 ppm
I20260810 06:46:37.031387 31567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:37 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:37.038693 31567 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:46:37.039988 31567 webserver.cc:460] Webserver started at http://127.10.242.62:44257/ using document root <none> and password file <none>
I20260810 06:46:37.040576 31567 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:37.040669 31567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:37.040959 31567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:46:37 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:37.096998 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "fc2be006faba40ff8464aab72672ea0f"
format_stamp: "Formatted at 2026-08-10 06:46:37 on dist-test-slave-zj8q"
server_key: "bf63f68db9cad1c1c54c59f7b66ce025"
server_key_iv: "55015f28aedc50813364a6c2ad3690c2"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:37.097910 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "fc2be006faba40ff8464aab72672ea0f"
format_stamp: "Formatted at 2026-08-10 06:46:37 on dist-test-slave-zj8q"
server_key: "bf63f68db9cad1c1c54c59f7b66ce025"
server_key_iv: "55015f28aedc50813364a6c2ad3690c2"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:37.103343 31567 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:46:37.123489 31594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:37.125088 31567 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:46:37.125279 31567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "fc2be006faba40ff8464aab72672ea0f"
format_stamp: "Formatted at 2026-08-10 06:46:37 on dist-test-slave-zj8q"
server_key: "bf63f68db9cad1c1c54c59f7b66ce025"
server_key_iv: "55015f28aedc50813364a6c2ad3690c2"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:37.125509 31567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:37.152705 31567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:37.156359 31567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:37.156665 31567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:37.174328 31567 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:42711
I20260810 06:46:37.174360 31684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:42711 every 8 connection(s)
I20260810 06:46:37.175606 31567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:46:37.179270 31685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:37.183351 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 31567
I20260810 06:46:37.183583 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:46:37.184430 11208 external_mini_cluster.cc:1428] Setting key 9549dca793e0fbebef6673dd9c46ca0f
I20260810 06:46:37.190734 31685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Bootstrap starting.
I20260810 06:46:37.194211 31685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:37.195309 31685 log.cc:824] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:37.198444 31685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: No bootstrap required, opened a new log
Aug 10 06:46:37 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:37.206061 31685 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } }
I20260810 06:46:37.206496 31685 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:37.206593 31685 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc2be006faba40ff8464aab72672ea0f, State: Initialized, Role: FOLLOWER
I20260810 06:46:37.206701 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:37.186494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.10.242.62:42711)
0810 06:46:37.187218 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:37.187229 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:37.187270 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:46:37.189237 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:37.189259 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:37.189621 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:37.189917 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:37.191598 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:37.192899 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:37.194676 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:37.195330 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:37.195494 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:37.199768 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:37.199775 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:37.199792 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:37.199837 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:37.202763 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:37.203450 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:37.203457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:37.203462 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:37.203616 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:37.204001 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:37.204005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:37.204007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:37.204567 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:37.204899 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:37.205325 (+ 426us) server_negotiation.cc:299] Negotiation successful
0810 06:46:37.205698 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":337,"threads_started":1}
I20260810 06:46:37.207341 31685 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } }
I20260810 06:46:37.207628 31685 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:46:37.207769 31685 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:46:37.207960 31685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:37.210819 31685 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } }
I20260810 06:46:37.211311 31685 leader_election.cc:302] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc2be006faba40ff8464aab72672ea0f; no voters:
I20260810 06:46:37.211855 31685 leader_election.cc:288] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:46:37.211964 31694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:37.212632 31694 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 1 LEADER]: Becoming Leader. State: Replica: fc2be006faba40ff8464aab72672ea0f, State: Running, Role: LEADER
I20260810 06:46:37.213101 31694 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } }
I20260810 06:46:37.213804 31685 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:37.215989 31697 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc2be006faba40ff8464aab72672ea0f. Latest consensus state: current_term: 1 leader_uuid: "fc2be006faba40ff8464aab72672ea0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } } }
I20260810 06:46:37.216393 31697 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:37.217465 31702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:46:37.217444 31696 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc2be006faba40ff8464aab72672ea0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } } }
I20260810 06:46:37.217780 31696 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:37.222743 31702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:46:37.223493 31685 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:46:37.242189 31702 catalog_manager.cc:1347] Generated new cluster ID: ca62cc9950e343b882fb5dbda908357b
I20260810 06:46:37.242334 31702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:46:37.259160 31702 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:46:37.261081 31702 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:46:37.277189 31702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Generated new TSK 0
I20260810 06:46:37.278323 31702 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:46:37.996986 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:46263/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42711
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:46:38 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 06:46:38.430505 31751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:38.430871 31751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:38.431084 31751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:38.442600 31751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:38.442816 31751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:38.443071 31751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:46:38.458961 31751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.242.20:46263/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:38.460678 31751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:38.462493 31751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:38.477102 31767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:38.477320 31769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:38.477588 31765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:38.478204 31751 server_base.cc:1034] running on GCE node
I20260810 06:46:38.478803 31751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:38.479748 31751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:38.480958 31751 hybrid_clock.cc:648] HybridClock initialized: now 1786344398480878 us; error 72 us; skew 500 ppm
I20260810 06:46:38.481268 31751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:38 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:38.484330 31751 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:46:38.485945 31751 webserver.cc:460] Webserver started at http://127.10.242.1:45833/ using document root <none> and password file <none>
I20260810 06:46:38.486663 31751 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:38.486774 31751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:38.487074 31751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:46:38 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:38.509193 31751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "f9a5e7c69e554f6594b6b387d99c8419"
format_stamp: "Formatted at 2026-08-10 06:46:38 on dist-test-slave-zj8q"
server_key: "90b642fb68db39970f5686bf85aa981e"
server_key_iv: "256f48e0ea4f5a3120a83e7714333d58"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:38.511750 31751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "f9a5e7c69e554f6594b6b387d99c8419"
format_stamp: "Formatted at 2026-08-10 06:46:38 on dist-test-slave-zj8q"
server_key: "90b642fb68db39970f5686bf85aa981e"
server_key_iv: "256f48e0ea4f5a3120a83e7714333d58"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:38.530506 31751 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.009s sys 0.008s
I20260810 06:46:38.555487 31778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:38.557375 31751 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:46:38.557670 31751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "f9a5e7c69e554f6594b6b387d99c8419"
format_stamp: "Formatted at 2026-08-10 06:46:38 on dist-test-slave-zj8q"
server_key: "90b642fb68db39970f5686bf85aa981e"
server_key_iv: "256f48e0ea4f5a3120a83e7714333d58"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:38.558014 31751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:38.624984 31751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:38.630318 31751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:38.630707 31751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:38.650620 31751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:38.658932 31751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:38.659119 31751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 06:46:38.659320 31751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:38.659430 31751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:46:38 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:38.977648 31751 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:44715
I20260810 06:46:38.977672 31970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:44715 every 8 connection(s)
I20260810 06:46:38.979481 31751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:46:38.985729 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 31751
I20260810 06:46:38.985941 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:46:38.986824 11208 external_mini_cluster.cc:1428] Setting key ba9c68d142f113bd257cac95af80b234
Aug 10 06:46:39 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
I20260810 06:46:39.026651 31976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:38.983059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42711 (local address 127.10.242.1:52735)
0810 06:46:39.002008 (+ 18949us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:39.002087 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:39.004511 (+ 2424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:39.006931 (+ 2420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:39.006944 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:39.007590 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:39.008429 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:39.008455 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:39.010061 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:39.010066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:39.012256 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:39.012281 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:39.013516 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:39.013525 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:39.013826 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:39.014671 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:39.014703 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:39.017613 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:39.021693 (+ 4080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:39.021702 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:39.021721 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:39.022323 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:39.022791 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:39.022795 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:39.022798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:39.023117 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:39.023864 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:39.023872 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:39.024273 (+ 401us) client_negotiation.cc:769] Sending connection context
0810 06:46:39.024602 (+ 329us) client_negotiation.cc:240] Negotiation successful
0810 06:46:39.025136 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18587,"thread_start_us":21767,"threads_started":1}
I20260810 06:46:39.027787 31975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:38.982490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:52735 (local address 127.10.242.62:42711)
0810 06:46:39.006142 (+ 23652us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:39.006148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:39.006188 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:46:39.006251 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:39.006257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:39.006347 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:39.006462 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:39.008643 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:39.009846 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:39.012444 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:39.013077 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:39.013199 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:39.017869 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:39.017877 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:39.017882 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:39.017923 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:39.021508 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:39.022470 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:39.022475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:39.022479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:39.022650 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:39.023259 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:39.023264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:39.023268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:39.023631 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:39.023776 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:39.027191 (+ 3415us) server_negotiation.cc:299] Negotiation successful
0810 06:46:39.027407 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23502,"thread_start_us":23344,"threads_started":1}
I20260810 06:46:39.029667 31972 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42711
I20260810 06:46:39.030023 31972 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:39.031085 31972 heartbeater.cc:511] Master 127.10.242.62:42711 requested a full tablet report, sending...
I20260810 06:46:39.033737 31617 ts_manager.cc:194] Registered new tserver with Master: f9a5e7c69e554f6594b6b387d99c8419 (127.10.242.1:44715)
I20260810 06:46:39.035846 31617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:52735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:46:39.062676 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:46263/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42711
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:46:39 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344398, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
W20260810 06:46:39.502511 31980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:39.502831 31980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:39.502965 31980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:39.510471 31980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:39.510576 31980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:39.510732 31980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:46:39.520609 31980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.242.20:46263/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:39.521955 31980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:39.523386 31980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:39.536722 31999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:39.536695 32000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:39.537743 31980 server_base.cc:1034] running on GCE node
W20260810 06:46:39.537811 32005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:39.538558 31980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:39.539513 31980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:39.540737 31980 hybrid_clock.cc:648] HybridClock initialized: now 1786344399540679 us; error 48 us; skew 500 ppm
I20260810 06:46:39.541026 31980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:39 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344399, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:39.544250 31980 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:46:39.545717 31980 webserver.cc:460] Webserver started at http://127.10.242.2:33543/ using document root <none> and password file <none>
I20260810 06:46:39.546422 31980 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:39.546551 31980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:39.546854 31980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:46:39 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344399, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:39.567466 31980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "41e8864b8124421590155dff124dfddc"
format_stamp: "Formatted at 2026-08-10 06:46:39 on dist-test-slave-zj8q"
server_key: "3a7d912cc3e460ddbbf3f7172f80e277"
server_key_iv: "279bceafa12ca1d4fc9330d65646ed6f"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:39.568715 31980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "41e8864b8124421590155dff124dfddc"
format_stamp: "Formatted at 2026-08-10 06:46:39 on dist-test-slave-zj8q"
server_key: "3a7d912cc3e460ddbbf3f7172f80e277"
server_key_iv: "279bceafa12ca1d4fc9330d65646ed6f"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:39.575136 31980 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260810 06:46:39.600874 32021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:39.602454 31980 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.000s
I20260810 06:46:39.602663 31980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "41e8864b8124421590155dff124dfddc"
format_stamp: "Formatted at 2026-08-10 06:46:39 on dist-test-slave-zj8q"
server_key: "3a7d912cc3e460ddbbf3f7172f80e277"
server_key_iv: "279bceafa12ca1d4fc9330d65646ed6f"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:39.602901 31980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:39.620420 31980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:39.624446 31980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:39.624771 31980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:39.625864 31980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:39.627431 31980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:39.627588 31980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 06:46:39.627749 31980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:39.627807 31980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:39.669682 31980 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:35873
I20260810 06:46:39.669716 32198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:35873 every 8 connection(s)
I20260810 06:46:39.671156 31980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:46:39.677078 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 31980
I20260810 06:46:39.677263 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:46:39.677909 11208 external_mini_cluster.cc:1428] Setting key 1057bb06e9ce4af791d9dd3d05aac85d
Aug 10 06:46:39 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344399, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:39.702826 32202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:39.675428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:53937 (local address 127.10.242.62:42711)
0810 06:46:39.682782 (+ 7354us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:39.682789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:39.682813 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:46:39.682878 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:39.682884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:39.682957 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:39.683225 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:39.685167 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:39.686380 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:39.689908 (+ 3528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:39.690574 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:39.690717 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:39.695426 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:39.695433 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:39.695438 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:39.695477 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:39.698931 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:39.699887 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:39.699893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:39.699897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:39.700050 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:39.700624 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:39.700629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:39.700652 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:39.701021 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:39.701146 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:39.702144 (+ 998us) server_negotiation.cc:299] Negotiation successful
0810 06:46:39.702424 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7210,"thread_start_us":7258,"threads_started":1}
I20260810 06:46:39.705382 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:39.676269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42711 (local address 127.10.242.2:53937)
0810 06:46:39.678702 (+ 2433us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:39.678763 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:39.681401 (+ 2638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:39.683494 (+ 2093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:39.683507 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:39.684148 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:39.684944 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:39.684970 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:39.686576 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:39.686582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:39.689729 (+ 3147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:39.689746 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:39.691019 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:39.691026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:39.691328 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:39.692077 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:39.692108 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:39.695127 (+ 3019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:39.699108 (+ 3981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:39.699118 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:39.699138 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:39.699744 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:39.700193 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:39.700198 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:39.700202 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:39.700490 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:39.701260 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:39.701268 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:39.701694 (+ 426us) client_negotiation.cc:769] Sending connection context
0810 06:46:39.703367 (+ 1673us) client_negotiation.cc:240] Negotiation successful
0810 06:46:39.703941 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2092,"thread_start_us":2470,"threads_started":1}
I20260810 06:46:39.707335 32199 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42711
I20260810 06:46:39.707669 32199 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:39.708585 32199 heartbeater.cc:511] Master 127.10.242.62:42711 requested a full tablet report, sending...
I20260810 06:46:39.710335 31617 ts_manager.cc:194] Registered new tserver with Master: 41e8864b8124421590155dff124dfddc (127.10.242.2:35873)
I20260810 06:46:39.711133 31617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:53937
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:46:39.751849 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:46263/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42711
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:46:40.039587 31972 heartbeater.cc:503] Master 127.10.242.62:42711 was elected leader, sending a full tablet report...
W20260810 06:46:40.083633 32211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:40.083922 32211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:40.084077 32211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:40.091113 32211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:40.091223 32211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:40.091383 32211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:46:40.100976 32211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.242.20:46263/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:40.102250 32211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:40.103533 32211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:40.116695 32220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:40.116659 32223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:40.116748 32219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:40.117210 32211 server_base.cc:1034] running on GCE node
I20260810 06:46:40.117805 32211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:40.118754 32211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:40.119956 32211 hybrid_clock.cc:648] HybridClock initialized: now 1786344400119910 us; error 37 us; skew 500 ppm
I20260810 06:46:40.120275 32211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:40 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344400, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:40.123314 32211 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:46:40.124712 32211 webserver.cc:460] Webserver started at http://127.10.242.3:37077/ using document root <none> and password file <none>
I20260810 06:46:40.125391 32211 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:40.125527 32211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:40.125834 32211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:46:40 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344400, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:40.142725 32211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "acb39e30cc9f4e1788aefd52b0a873c9"
format_stamp: "Formatted at 2026-08-10 06:46:40 on dist-test-slave-zj8q"
server_key: "95adcbe68c2ce9e699b9842b4447948d"
server_key_iv: "b11254a1df7a4477753c80af564ce5c1"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:40.143615 32211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "acb39e30cc9f4e1788aefd52b0a873c9"
format_stamp: "Formatted at 2026-08-10 06:46:40 on dist-test-slave-zj8q"
server_key: "95adcbe68c2ce9e699b9842b4447948d"
server_key_iv: "b11254a1df7a4477753c80af564ce5c1"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:40.149448 32211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:46:40.167521 32231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:40.169268 32211 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 06:46:40.169442 32211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "acb39e30cc9f4e1788aefd52b0a873c9"
format_stamp: "Formatted at 2026-08-10 06:46:40 on dist-test-slave-zj8q"
server_key: "95adcbe68c2ce9e699b9842b4447948d"
server_key_iv: "b11254a1df7a4477753c80af564ce5c1"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:40.169668 32211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:40.188205 32211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:40.192050 32211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:40.192340 32211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:40.193287 32211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:40.194667 32211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:40.194746 32211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:40.194864 32211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:40.194921 32211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:40.231637 32211 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:46651
I20260810 06:46:40.231724 32426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:46651 every 8 connection(s)
I20260810 06:46:40.233028 32211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:46:40.240497 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 32211
I20260810 06:46:40.240679 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:46:40.241155 11208 external_mini_cluster.cc:1428] Setting key bf87e1cca606c3ccb393ae016e6dbea7
Aug 10 06:46:40 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344400, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:40.256434 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:40.235809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:59859 (local address 127.10.242.62:42711)
0810 06:46:40.236441 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:40.236448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:40.237393 (+ 945us) server_negotiation.cc:407] Connection header received
0810 06:46:40.239961 (+ 2568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:40.239965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:40.240032 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:40.240128 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:40.242051 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.243097 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.245709 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.246114 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.246263 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:40.250115 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:40.250120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:40.250124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:40.250184 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:40.252740 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:40.253443 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:40.253449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:40.253452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:40.253581 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:40.254036 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:40.254040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:40.254042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:40.254357 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:40.254515 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:40.255474 (+ 959us) server_negotiation.cc:299] Negotiation successful
0810 06:46:40.255856 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":286,"threads_started":1}
I20260810 06:46:40.257206 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:40.236320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42711 (local address 127.10.242.3:59859)
0810 06:46:40.237205 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:40.237259 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:40.239673 (+ 2414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:40.240424 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:40.240436 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:40.241045 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:40.241873 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:40.241900 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.243295 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.243300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:40.245546 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:40.245563 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.246363 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.246371 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:40.246683 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:40.247462 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:40.247493 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:40.249897 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:40.252871 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:40.252880 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:40.252896 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:40.253330 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:40.253694 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:40.253699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:40.253703 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:40.253925 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:40.254536 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:40.254544 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:40.254946 (+ 402us) client_negotiation.cc:769] Sending connection context
0810 06:46:40.255305 (+ 359us) client_negotiation.cc:240] Negotiation successful
0810 06:46:40.255828 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":339,"threads_started":1}
I20260810 06:46:40.258805 32428 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42711
I20260810 06:46:40.259138 32428 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:40.259992 32428 heartbeater.cc:511] Master 127.10.242.62:42711 requested a full tablet report, sending...
I20260810 06:46:40.261533 31617 ts_manager.cc:194] Registered new tserver with Master: acb39e30cc9f4e1788aefd52b0a873c9 (127.10.242.3:46651)
I20260810 06:46:40.262333 31617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:59859
I20260810 06:46:40.268374 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:46:40.270030 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 31751
Aug 10 06:46:40 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:46:40.288200 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:40.271039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39514 (local address 127.10.242.1:44715)
0810 06:46:40.271664 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:40.271670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:40.271694 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:46:40.271755 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:40.271761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:40.271928 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:40.272042 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:40.274011 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.275183 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.277202 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.277765 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.277859 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:40.282661 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:40.282673 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:40.282687 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:40.282722 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:40.285420 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:40.286076 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:40.286083 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:40.286085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:40.286275 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:40.286642 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:40.286647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:40.286650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:40.286981 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:40.287274 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:40.287592 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 06:46:40.287864 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":319,"threads_started":1}
I20260810 06:46:40.289589 31962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:40.371119 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 31751
I20260810 06:46:40.397886 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 31980
Aug 10 06:46:40 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:46:40.413659 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:40.398776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.10.242.2:35873)
0810 06:46:40.399340 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:40.399347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:40.399374 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:46:40.399436 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:40.399443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:40.399650 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:40.399801 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:40.401139 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.402188 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.403503 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.404071 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.404189 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:40.408339 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:40.408351 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:40.408366 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:40.408398 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:40.410938 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:40.411612 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:40.411619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:40.411622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:40.411763 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:40.412110 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:40.412113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:40.412115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:40.412444 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:40.412744 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:40.413061 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 06:46:40.413319 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":281,"threads_started":1}
I20260810 06:46:40.414990 32193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:40.495931 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 31980
I20260810 06:46:40.518501 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 32211
Aug 10 06:46:40 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:46:40.534525 32447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:40.519453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.10.242.3:46651)
0810 06:46:40.520143 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:40.520152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:40.520181 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:46:40.520245 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:40.520251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:40.520464 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:40.520612 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:40.521979 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.523060 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.524306 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:40.524919 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:40.525034 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:40.529365 (+ 4331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:40.529377 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:40.529393 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:40.529430 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:40.532059 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:40.532763 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:40.532771 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:40.532773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:40.532904 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:40.533200 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:40.533203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:40.533205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:40.533517 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:40.533772 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:40.533934 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 06:46:40.534136 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":385,"threads_started":1}
I20260810 06:46:40.535879 32419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:40.616927 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 32211
I20260810 06:46:40.641937 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 31567
I20260810 06:46:40.643225 31674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:40.748288 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 31567
I20260810 06:46:40.785480 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:42711
--webserver_interface=127.10.242.62
--webserver_port=44257
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:42711
--ranger_config_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:46263/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.242.20:46263/kms with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:41.104894 32453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:41.105165 32453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:41.105259 32453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:41.112404 32453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:41.112507 32453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:41.112560 32453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:41.112622 32453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:41.112669 32453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:41.122680 32453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.242.20:46263/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:42711
--ranger_config_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:42711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=44257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:41.123973 32453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:41.125339 32453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:41.137166 32459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:41.137198 32460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:41.137323 32466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:41.138377 32453 server_base.cc:1034] running on GCE node
I20260810 06:46:41.139243 32453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:41.140642 32453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:41.141848 32453 hybrid_clock.cc:648] HybridClock initialized: now 1786344401141799 us; error 41 us; skew 500 ppm
I20260810 06:46:41.142220 32453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:41 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344401, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:41.145183 32453 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:46:41.146703 32453 webserver.cc:460] Webserver started at http://127.10.242.62:44257/ using document root <none> and password file <none>
I20260810 06:46:41.147346 32453 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:41.147469 32453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:46:41 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344401, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:41.225421 32453 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:46:41.228490 32483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:41.229859 32453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:46:41.230033 32453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "fc2be006faba40ff8464aab72672ea0f"
format_stamp: "Formatted at 2026-08-10 06:46:37 on dist-test-slave-zj8q"
server_key: "bf63f68db9cad1c1c54c59f7b66ce025"
server_key_iv: "55015f28aedc50813364a6c2ad3690c2"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:41.231058 32453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:41.260509 32453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:41.264032 32453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:41.264293 32453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:41.281054 32453 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:42711
I20260810 06:46:41.281087 32572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:42711 every 8 connection(s)
I20260810 06:46:41.282416 32453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:46:41.288873 32574 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:46:41.289757 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 32453
I20260810 06:46:41.290606 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:46263/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:44715
--local_ip_for_outbound_sockets=127.10.242.1
--tserver_master_addrs=127.10.242.62:42711
--webserver_port=45833
--webserver_interface=127.10.242.1
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.242.20:46263/kms with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:46:41.292060 32574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Bootstrap starting.
I20260810 06:46:41.309923 32574 log.cc:824] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:41.318737 32574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:46:41.319255 32574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Bootstrap complete.
I20260810 06:46:41.327915 32574 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } }
I20260810 06:46:41.328750 32574 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc2be006faba40ff8464aab72672ea0f, State: Initialized, Role: FOLLOWER
I20260810 06:46:41.329348 32574 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } }
I20260810 06:46:41.329569 32574 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:46:41.329671 32574 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:46:41.329840 32574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:46:41.334762 32574 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } }
I20260810 06:46:41.335295 32574 leader_election.cc:302] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc2be006faba40ff8464aab72672ea0f; no voters:
I20260810 06:46:41.335939 32574 leader_election.cc:288] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:46:41.336081 32581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:46:41.337154 32581 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [term 2 LEADER]: Becoming Leader. State: Replica: fc2be006faba40ff8464aab72672ea0f, State: Running, Role: LEADER
I20260810 06:46:41.337756 32581 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } }
I20260810 06:46:41.338056 32574 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:41.340385 32584 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc2be006faba40ff8464aab72672ea0f. Latest consensus state: current_term: 2 leader_uuid: "fc2be006faba40ff8464aab72672ea0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } } }
I20260810 06:46:41.341274 32584 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:41.340672 32583 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fc2be006faba40ff8464aab72672ea0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc2be006faba40ff8464aab72672ea0f" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42711 } } }
I20260810 06:46:41.341552 32583 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:41.352418 32593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:46:41.361027 32593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:46:41.361838 32574 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:46:41.362497 32593 catalog_manager.cc:1259] Loaded cluster ID: ca62cc9950e343b882fb5dbda908357b
I20260810 06:46:41.362668 32593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:46:41.366590 32593 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:46:41.368024 32593 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Loaded TSK: 0
I20260810 06:46:41.369588 32593 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:46:41.728179 32577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:41.728664 32577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:41.728880 32577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:41.739168 32577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:41.739365 32577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:41.739643 32577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:46:41.753433 32577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.242.20:46263/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:44715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=45833
--tserver_master_addrs=127.10.242.62:42711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:41.754992 32577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:41.756454 32577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:41.788976 32647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:41.789302 32646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:41.790885 32651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:41.796641 32577 server_base.cc:1034] running on GCE node
I20260810 06:46:41.797319 32577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:41.799702 32577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:41.800988 32577 hybrid_clock.cc:648] HybridClock initialized: now 1786344401800921 us; error 46 us; skew 500 ppm
I20260810 06:46:41.801352 32577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:41 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344401, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:41.804595 32577 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:46:41.806438 32577 webserver.cc:460] Webserver started at http://127.10.242.1:45833/ using document root <none> and password file <none>
I20260810 06:46:41.807207 32577 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:41.807348 32577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:46:41 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344401, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:41.840224 32577 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.002s
I20260810 06:46:41.862757 32663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:41.864588 32577 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.005s sys 0.016s
I20260810 06:46:41.864846 32577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "f9a5e7c69e554f6594b6b387d99c8419"
format_stamp: "Formatted at 2026-08-10 06:46:38 on dist-test-slave-zj8q"
server_key: "90b642fb68db39970f5686bf85aa981e"
server_key_iv: "256f48e0ea4f5a3120a83e7714333d58"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:41.866442 32577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:41.912637 32577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:41.917899 32577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:41.918340 32577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:41.924429 32577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:41.930015 32577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:41.930246 32577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:41.930454 32577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:41.930611 32577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:41.989683 32577 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:44715
I20260810 06:46:41.989741 393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:44715 every 8 connection(s)
I20260810 06:46:41.991571 32577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:46:41.998644 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 32577
I20260810 06:46:41.999536 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:46263/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:35873
--local_ip_for_outbound_sockets=127.10.242.2
--tserver_master_addrs=127.10.242.62:42711
--webserver_port=33543
--webserver_interface=127.10.242.2
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.242.20:46263/kms with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:46:42 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344401, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:42.050313 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:41.997632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42711 (local address 127.10.242.1:45873)
0810 06:46:42.021733 (+ 24101us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:42.021794 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:42.024401 (+ 2607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:42.026414 (+ 2013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:42.026426 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:42.027103 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:42.027874 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:42.027911 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:42.030203 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:42.030209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:42.032299 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:42.032316 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:42.033799 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:42.033807 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:42.034118 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:42.035115 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:42.035153 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:42.037726 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:42.045147 (+ 7421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:42.045156 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:42.045174 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:42.045777 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:42.046345 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:42.046350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:42.046354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:42.046593 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:42.047694 (+ 1101us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:42.047703 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:42.048101 (+ 398us) client_negotiation.cc:769] Sending connection context
0810 06:46:42.048395 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 06:46:42.048920 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23736,"thread_start_us":24448,"threads_started":1}
I20260810 06:46:42.052439 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:41.997613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:45873 (local address 127.10.242.62:42711)
0810 06:46:42.021053 (+ 23440us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:42.021078 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:42.021977 (+ 899us) server_negotiation.cc:407] Connection header received
0810 06:46:42.025156 (+ 3179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:42.025184 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:42.025680 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:42.026083 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:42.028123 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:42.029987 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:42.032469 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:42.033517 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:42.033735 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:42.038047 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:42.038059 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:42.038080 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:42.038128 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:42.044963 (+ 6835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:42.045949 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:42.045959 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:42.045967 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:42.046205 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:42.046721 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:42.046726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:42.046730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:42.047391 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:42.047615 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:42.050792 (+ 3177us) server_negotiation.cc:299] Negotiation successful
0810 06:46:42.051193 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23115,"thread_start_us":22879,"threads_started":1}
I20260810 06:46:42.056099 394 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42711
I20260810 06:46:42.056437 394 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:42.057310 394 heartbeater.cc:511] Master 127.10.242.62:42711 requested a full tablet report, sending...
W20260810 06:46:42.060672 394 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:42711 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:46:42.060901 394 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:42.061169 394 heartbeater.cc:511] Master 127.10.242.62:42711 requested a full tablet report, sending...
I20260810 06:46:42.062425 394 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:42.062676 394 heartbeater.cc:511] Master 127.10.242.62:42711 requested a full tablet report, sending...
W20260810 06:46:42.063771 394 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:46:42.391877 401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:42.392122 401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:42.392231 401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:42.400316 401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:42.400497 401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:42.400688 401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:46:42.410980 401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.242.20:46263/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:35873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=33543
--tserver_master_addrs=127.10.242.62:42711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:42.412540 401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:42.414294 401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:42.435843 421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:42.436434 422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:42.437630 424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:42.437951 401 server_base.cc:1034] running on GCE node
I20260810 06:46:42.438798 401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:42.439970 401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:42.441264 401 hybrid_clock.cc:648] HybridClock initialized: now 1786344402441195 us; error 54 us; skew 500 ppm
I20260810 06:46:42.441638 401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:42 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:42.445777 401 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:46:42.447597 401 webserver.cc:460] Webserver started at http://127.10.242.2:33543/ using document root <none> and password file <none>
I20260810 06:46:42.448515 401 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:42.448743 401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:46:42 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:42.478056 401 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.004s sys 0.000s
I20260810 06:46:42.488394 441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:42.490253 401 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.009s sys 0.000s
I20260810 06:46:42.490491 401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "41e8864b8124421590155dff124dfddc"
format_stamp: "Formatted at 2026-08-10 06:46:39 on dist-test-slave-zj8q"
server_key: "3a7d912cc3e460ddbbf3f7172f80e277"
server_key_iv: "279bceafa12ca1d4fc9330d65646ed6f"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:42.492002 401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:42.523344 401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:42.531270 401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:42.531581 401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:42.532823 401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:42.542996 401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:42.543124 401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:42.543236 401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:42.543322 401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:46:42 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:42.625728 401 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:35873
I20260810 06:46:42.626015 634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:35873 every 8 connection(s)
I20260810 06:46:42.627432 401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:46:42.632718 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 401
I20260810 06:46:42.633462 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.242.20:46263/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:46651
--local_ip_for_outbound_sockets=127.10.242.3
--tserver_master_addrs=127.10.242.62:42711
--webserver_port=37077
--webserver_interface=127.10.242.3
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.242.20:46263/kms with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:46:42 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:42.665017 639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:42.631233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:53195 (local address 127.10.242.62:42711)
0810 06:46:42.638501 (+ 7268us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:42.638507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:42.638528 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:46:42.638594 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:42.638599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:42.638680 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:42.638786 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:42.640899 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:42.642107 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:42.644537 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:42.645213 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:42.645309 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:42.656808 (+ 11499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:42.656816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:42.656821 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:42.656864 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:42.661624 (+ 4760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:42.662399 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:42.662405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:42.662409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:42.662540 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:42.663019 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:42.663024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:42.663027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:42.663401 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:42.663539 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:42.664544 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0810 06:46:42.664758 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7148,"thread_start_us":7027,"threads_started":1}
I20260810 06:46:42.667091 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:42.632287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42711 (local address 127.10.242.2:53195)
0810 06:46:42.635237 (+ 2950us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:42.635293 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:42.637770 (+ 2477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:42.639254 (+ 1484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:42.639265 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:42.639901 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:42.640689 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:42.640714 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:42.642360 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:42.642365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:42.644372 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:42.644388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:42.645709 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:42.645717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:42.645991 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:42.650497 (+ 4506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:42.650535 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:42.656489 (+ 5954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:42.661786 (+ 5297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:42.661794 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:42.661808 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:42.662283 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:42.662656 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:42.662660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:42.662662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:42.662871 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:42.663627 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:42.663635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:42.664101 (+ 466us) client_negotiation.cc:769] Sending connection context
0810 06:46:42.665318 (+ 1217us) client_negotiation.cc:240] Negotiation successful
0810 06:46:42.665789 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2661,"thread_start_us":2444,"threads_started":1}
I20260810 06:46:42.669078 637 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42711
I20260810 06:46:42.669425 637 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:42.670349 637 heartbeater.cc:511] Master 127.10.242.62:42711 requested a full tablet report, sending...
I20260810 06:46:42.672603 32518 ts_manager.cc:194] Registered new tserver with Master: 41e8864b8124421590155dff124dfddc (127.10.242.2:35873)
I20260810 06:46:42.674789 32518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:53195
Aug 10 06:46:42 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:43.063928 394 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:43.064385 394 heartbeater.cc:511] Master 127.10.242.62:42711 requested a full tablet report, sending...
I20260810 06:46:43.065698 32518 ts_manager.cc:194] Registered new tserver with Master: f9a5e7c69e554f6594b6b387d99c8419 (127.10.242.1:44715)
I20260810 06:46:43.066905 32518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:45873
W20260810 06:46:43.105014 642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:43.105512 642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:43.105757 642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:43.117115 642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:43.117246 642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:43.117476 642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:46:43.132871 642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.242.20:46263/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:46651
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=37077
--tserver_master_addrs=127.10.242.62:42711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:43.134644 642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:43.136340 642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:43.152699 649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:43.153774 651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:43.162400 642 server_base.cc:1034] running on GCE node
W20260810 06:46:43.163242 653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:43.164041 642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:43.165285 642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:43.166806 642 hybrid_clock.cc:648] HybridClock initialized: now 1786344403166722 us; error 69 us; skew 500 ppm
I20260810 06:46:43.167183 642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:43 dist-test-slave-zj8q krb5kdc[30856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344403, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:43.171653 642 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:46:43.173782 642 webserver.cc:460] Webserver started at http://127.10.242.3:37077/ using document root <none> and password file <none>
I20260810 06:46:43.174811 642 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:43.174978 642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:46:43 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344403, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:43.203558 642 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:46:43.207540 668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:43.209466 642 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:46:43.209651 642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "acb39e30cc9f4e1788aefd52b0a873c9"
format_stamp: "Formatted at 2026-08-10 06:46:40 on dist-test-slave-zj8q"
server_key: "95adcbe68c2ce9e699b9842b4447948d"
server_key_iv: "b11254a1df7a4477753c80af564ce5c1"
server_key_version: "kuduclusterkey@0"
I20260810 06:46:43.211119 642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:43.229825 642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:43.236933 642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:43.237231 642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:43.250588 642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:43.252637 642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:43.252791 642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:43.252911 642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:43.252993 642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:46:43 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344402, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for HTTP/127.10.242.20@KRBTEST.COM
I20260810 06:46:43.476996 864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:46651 every 8 connection(s)
I20260810 06:46:43.477324 642 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:46651
I20260810 06:46:43.479312 642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:46:43.482383 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 642
Aug 10 06:46:43 dist-test-slave-zj8q krb5kdc[30856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344403, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:43.518077 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.482412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:48479 (local address 127.10.242.62:42711)
0810 06:46:43.490553 (+ 8141us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.490561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.498697 (+ 8136us) server_negotiation.cc:407] Connection header received
0810 06:46:43.501527 (+ 2830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.501533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.501650 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.501778 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:43.504014 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.505130 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.506781 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.507436 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.507563 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.511537 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:43.511544 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:43.511548 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:43.511603 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:43.514521 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.515271 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.515277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.515281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.515390 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.515898 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.515904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.515907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.516354 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:43.516546 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.517454 (+ 908us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.517669 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":7998,"thread_start_us":7872,"threads_started":1}
I20260810 06:46:43.518201 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.483340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.10.242.62:42711)
0810 06:46:43.501985 (+ 18645us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.501990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.502004 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:46:43.502043 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.502046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.502104 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.502230 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:43.503772 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.504907 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.506974 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.507714 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.507818 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.510591 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:43.510598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:43.510603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:43.510640 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:43.515165 (+ 4525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.516211 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.516215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.516217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.516336 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.516698 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.516703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.516705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.516950 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:43.517267 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.517539 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.517794 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":18526,"thread_start_us":18377,"threads_started":1}
I20260810 06:46:43.520181 881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.483051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42711 (local address 127.10.242.3:48479)
0810 06:46:43.498498 (+ 15447us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:43.498559 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:43.501177 (+ 2618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:43.502316 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:43.502328 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:43.502967 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:43.503801 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.503827 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.505279 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.505285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.506648 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.506662 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.507700 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.507708 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.507987 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.508816 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:43.508851 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:43.511285 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:43.514704 (+ 3419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:43.514713 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:43.514728 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:43.515145 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:43.515490 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:43.515495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:43.515498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:43.515769 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:43.516612 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:43.516619 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:43.517005 (+ 386us) client_negotiation.cc:769] Sending connection context
0810 06:46:43.518391 (+ 1386us) client_negotiation.cc:240] Negotiation successful
0810 06:46:43.518902 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15109,"thread_start_us":14851,"threads_started":1}
I20260810 06:46:43.521633 869 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42711
I20260810 06:46:43.521914 869 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:43.522748 869 heartbeater.cc:511] Master 127.10.242.62:42711 requested a full tablet report, sending...
I20260810 06:46:43.524154 32518 ts_manager.cc:194] Registered new tserver with Master: acb39e30cc9f4e1788aefd52b0a873c9 (127.10.242.3:46651)
I20260810 06:46:43.524887 32518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:48479
I20260810 06:46:43.533022 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:46:43.548209 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.536050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.10.242.62:42711)
0810 06:46:43.536220 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.536224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.536741 (+ 517us) server_negotiation.cc:407] Connection header received
0810 06:46:43.536913 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.536919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.536973 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.537047 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:43.538622 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.539382 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.540666 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.541080 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.541192 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.543676 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:43.543681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:43.543685 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:43.543713 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:43.546130 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.546801 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.546805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.546807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.546920 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.547211 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.547214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.547216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.547455 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:43.547635 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.547838 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.547987 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:46:43.552699 32518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:46:43.559973 32518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:46:43.594895 904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.573431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:46651 (local address 127.0.0.1:59660)
0810 06:46:43.584410 (+ 10979us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:43.584432 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:43.584571 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:43.585219 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:43.585227 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:43.585261 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:43.585747 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.585760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.589292 (+ 3532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.589297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.591954 (+ 2657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.591968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.593854 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.593860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.594038 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.594141 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 06:46:43.594257 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 06:46:43.594463 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10845,"mutex_wait_us":179,"thread_start_us":11241,"threads_started":1}
I20260810 06:46:43.594900 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.574216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.10.242.3:46651)
0810 06:46:43.583070 (+ 8854us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.583080 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.584519 (+ 1439us) server_negotiation.cc:407] Connection header received
0810 06:46:43.584739 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.584748 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.585006 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.585209 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:43.585938 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.587697 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.592151 (+ 4454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.593604 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.594101 (+ 497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.594357 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.594429 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.594572 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8666,"thread_start_us":8501,"threads_started":1}
I20260810 06:46:43.594923 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.573110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.10.242.2:35873)
0810 06:46:43.586347 (+ 13237us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.586357 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.586385 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:46:43.586485 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.586493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.586734 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.587579 (+ 845us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:43.587845 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.589102 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.592170 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.593577 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.593724 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.593928 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.594422 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.594588 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13053,"thread_start_us":14681,"threads_started":1}
I20260810 06:46:43.595505 912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.575799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:44715 (local address 127.0.0.1:57998)
0810 06:46:43.588381 (+ 12582us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:43.588405 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:43.588511 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:43.589143 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:43.589147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:43.589173 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:43.589989 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.589999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.591511 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.591516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.593037 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.593049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.594797 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.594803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.594971 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.594999 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 06:46:43.595084 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:46:43.595201 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12477,"thread_start_us":12342,"threads_started":1}
I20260810 06:46:43.596112 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.576094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.10.242.1:44715)
0810 06:46:43.588733 (+ 12639us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.588738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.588758 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:46:43.588832 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.588836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.588975 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.589164 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:43.590212 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.591389 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.593177 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.594325 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.594447 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.594640 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.595732 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.595851 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12503,"thread_start_us":12304,"threads_started":1}
I20260810 06:46:43.596812 902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.572874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:35873 (local address 127.0.0.1:33374)
0810 06:46:43.574009 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:43.574055 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:43.574239 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:43.586951 (+ 12712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:43.586957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:43.586984 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:43.587387 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.587400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.589256 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.589261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.591955 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.591968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.593818 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.593825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.594034 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.594113 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 06:46:43.594263 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 06:46:43.596509 (+ 2246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":2180,"thread_start_us":2094,"threads_started":1}
I20260810 06:46:43.599361 755 tablet_service.cc:1467] Processing CreateTablet for tablet a54d2e87a8484fa1b6fa71352b945963 (DEFAULT_TABLE table=test-table [id=9d2a28bf701548208bb9e549ce77953e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:43.600180 32746 tablet_service.cc:1467] Processing CreateTablet for tablet a54d2e87a8484fa1b6fa71352b945963 (DEFAULT_TABLE table=test-table [id=9d2a28bf701548208bb9e549ce77953e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:43.600174 527 tablet_service.cc:1467] Processing CreateTablet for tablet a54d2e87a8484fa1b6fa71352b945963 (DEFAULT_TABLE table=test-table [id=9d2a28bf701548208bb9e549ce77953e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:43.600829 755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54d2e87a8484fa1b6fa71352b945963. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:43.601434 527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54d2e87a8484fa1b6fa71352b945963. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:43.601461 32746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a54d2e87a8484fa1b6fa71352b945963. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:43.612231 917 tablet_bootstrap.cc:492] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: Bootstrap starting.
I20260810 06:46:43.615362 917 tablet_bootstrap.cc:654] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:43.616343 917 log.cc:824] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:43.617146 920 tablet_bootstrap.cc:492] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: Bootstrap starting.
I20260810 06:46:43.619158 918 tablet_bootstrap.cc:492] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: Bootstrap starting.
I20260810 06:46:43.620718 920 tablet_bootstrap.cc:654] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:43.621953 920 log.cc:824] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:43.624444 917 tablet_bootstrap.cc:492] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: No bootstrap required, opened a new log
I20260810 06:46:43.624734 917 ts_tablet_manager.cc:1397] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20260810 06:46:43.622525 918 tablet_bootstrap.cc:654] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:43.629462 918 log.cc:824] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:43.635934 920 tablet_bootstrap.cc:492] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: No bootstrap required, opened a new log
I20260810 06:46:43.635689 917 raft_consensus.cc:348] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } }
I20260810 06:46:43.636166 917 raft_consensus.cc:374] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:43.636221 920 ts_tablet_manager.cc:1397] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.004s
I20260810 06:46:43.636283 917 raft_consensus.cc:729] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb39e30cc9f4e1788aefd52b0a873c9, State: Initialized, Role: FOLLOWER
I20260810 06:46:43.636889 917 consensus_queue.cc:260] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } }
I20260810 06:46:43.637964 869 heartbeater.cc:503] Master 127.10.242.62:42711 was elected leader, sending a full tablet report...
I20260810 06:46:43.638726 917 ts_tablet_manager.cc:1428] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260810 06:46:43.640163 918 tablet_bootstrap.cc:492] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: No bootstrap required, opened a new log
I20260810 06:46:43.640401 918 ts_tablet_manager.cc:1397] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.009s
I20260810 06:46:43.645649 920 raft_consensus.cc:348] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } }
I20260810 06:46:43.646039 920 raft_consensus.cc:374] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:43.646145 920 raft_consensus.cc:729] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a5e7c69e554f6594b6b387d99c8419, State: Initialized, Role: FOLLOWER
I20260810 06:46:43.646688 920 consensus_queue.cc:260] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } }
I20260810 06:46:43.647684 394 heartbeater.cc:503] Master 127.10.242.62:42711 was elected leader, sending a full tablet report...
I20260810 06:46:43.648135 920 ts_tablet_manager.cc:1428] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260810 06:46:43.648398 918 raft_consensus.cc:348] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } }
I20260810 06:46:43.648814 918 raft_consensus.cc:374] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:43.648929 918 raft_consensus.cc:729] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e8864b8124421590155dff124dfddc, State: Initialized, Role: FOLLOWER
I20260810 06:46:43.649434 918 consensus_queue.cc:260] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } }
I20260810 06:46:43.650401 637 heartbeater.cc:503] Master 127.10.242.62:42711 was elected leader, sending a full tablet report...
I20260810 06:46:43.651073 918 ts_tablet_manager.cc:1428] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260810 06:46:43.687181 924 raft_consensus.cc:482] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:46:43.687467 924 raft_consensus.cc:504] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } }
I20260810 06:46:43.689155 924 leader_election.cc:288] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41e8864b8124421590155dff124dfddc (127.10.242.2:35873), f9a5e7c69e554f6594b6b387d99c8419 (127.10.242.1:44715)
I20260810 06:46:43.695252 881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.689564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:35873 (local address 127.10.242.3:55739)
0810 06:46:43.689768 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:43.689788 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:43.689913 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:43.690244 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:43.690249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:43.690282 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:43.690497 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.690509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.691692 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.691695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.693447 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.693461 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.694733 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.694737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.694862 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.694886 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:46:43.694964 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:46:43.695053 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":1}
I20260810 06:46:43.695371 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.689610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:55739 (local address 127.10.242.2:35873)
0810 06:46:43.689802 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.689808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.689848 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:46:43.690052 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.690056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.690114 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.690221 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:43.690643 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.691542 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.693633 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.694592 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.694755 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.694824 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.695005 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.695131 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:46:43.696146 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.690110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:56535 (local address 127.10.242.1:44715)
0810 06:46:43.690757 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.690763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.690780 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:46:43.690865 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.690870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.690939 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.691023 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:43.691447 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.692194 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.694233 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.695313 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.695470 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.695534 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.695869 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.695973 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504}
I20260810 06:46:43.696295 927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.689940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:44715 (local address 127.10.242.3:56535)
0810 06:46:43.690664 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:43.690685 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:43.690777 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:43.691036 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:43.691039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:43.691054 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:43.691322 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.691329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.692339 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.692344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.694056 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:43.694067 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.695606 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.695610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:43.695725 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.695746 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 06:46:43.695816 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:46:43.695926 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":3,"thread_start_us":440,"threads_started":1}
I20260810 06:46:43.696357 559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54d2e87a8484fa1b6fa71352b945963" candidate_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e8864b8124421590155dff124dfddc" is_pre_election: true
I20260810 06:46:43.696909 559 raft_consensus.cc:2454] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb39e30cc9f4e1788aefd52b0a873c9 in term 0.
I20260810 06:46:43.697314 309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54d2e87a8484fa1b6fa71352b945963" candidate_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a5e7c69e554f6594b6b387d99c8419" is_pre_election: true
I20260810 06:46:43.697630 671 leader_election.cc:302] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e8864b8124421590155dff124dfddc, acb39e30cc9f4e1788aefd52b0a873c9; no voters:
I20260810 06:46:43.697762 309 raft_consensus.cc:2454] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb39e30cc9f4e1788aefd52b0a873c9 in term 0.
I20260810 06:46:43.697989 924 raft_consensus.cc:2781] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:46:43.698138 924 raft_consensus.cc:482] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:46:43.698272 924 raft_consensus.cc:3037] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:43.700443 924 raft_consensus.cc:504] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } }
I20260810 06:46:43.700947 924 leader_election.cc:288] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [CANDIDATE]: Term 1 election: Requested vote from peers 41e8864b8124421590155dff124dfddc (127.10.242.2:35873), f9a5e7c69e554f6594b6b387d99c8419 (127.10.242.1:44715)
I20260810 06:46:43.701330 559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54d2e87a8484fa1b6fa71352b945963" candidate_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41e8864b8124421590155dff124dfddc"
I20260810 06:46:43.701426 309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a54d2e87a8484fa1b6fa71352b945963" candidate_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9a5e7c69e554f6594b6b387d99c8419"
I20260810 06:46:43.701488 559 raft_consensus.cc:3037] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:43.701619 309 raft_consensus.cc:3037] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:43.703691 559 raft_consensus.cc:2454] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb39e30cc9f4e1788aefd52b0a873c9 in term 1.
I20260810 06:46:43.703809 309 raft_consensus.cc:2454] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acb39e30cc9f4e1788aefd52b0a873c9 in term 1.
I20260810 06:46:43.704074 671 leader_election.cc:302] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41e8864b8124421590155dff124dfddc, acb39e30cc9f4e1788aefd52b0a873c9; no voters:
I20260810 06:46:43.704358 924 raft_consensus.cc:2781] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:43.704841 924 raft_consensus.cc:686] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 1 LEADER]: Becoming Leader. State: Replica: acb39e30cc9f4e1788aefd52b0a873c9, State: Running, Role: LEADER
I20260810 06:46:43.705283 924 consensus_queue.cc:237] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } }
I20260810 06:46:43.709623 32517 catalog_manager.cc:5495] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to acb39e30cc9f4e1788aefd52b0a873c9 (127.10.242.3). New cstate: current_term: 1 leader_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acb39e30cc9f4e1788aefd52b0a873c9" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 46651 } health_report { overall_health: HEALTHY } } }
I20260810 06:46:43.746318 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.740919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.10.242.3:46651)
0810 06:46:43.741105 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.741112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.741206 (+ 94us) server_negotiation.cc:407] Connection header received
0810 06:46:43.741422 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.741426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.741503 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.741630 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:46:43.742016 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.742783 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.744210 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.744596 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.744701 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.745219 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:46:43.745723 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:46:43.745861 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.745957 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.746037 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 06:46:43.746495 395 tablet.cc:2369] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:46:43.748091 877 tablet.cc:2369] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:46:43.754369 309 raft_consensus.cc:1261] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [term 1 FOLLOWER]: Refusing update from remote peer acb39e30cc9f4e1788aefd52b0a873c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:43.754462 559 raft_consensus.cc:1261] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [term 1 FOLLOWER]: Refusing update from remote peer acb39e30cc9f4e1788aefd52b0a873c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:43.755247 924 consensus_queue.cc:1035] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9a5e7c69e554f6594b6b387d99c8419" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 44715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:43.755575 928 consensus_queue.cc:1035] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41e8864b8124421590155dff124dfddc" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 35873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:43.766052 933 mvcc.cc:205] Tried to move back new op lower bound from 7316866677768572928 to 7316866677580496896. Current Snapshot: MvccSnapshot[applied={T|T < 7316866677768572928}]
I20260810 06:46:43.779243 32518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:43.779482 32518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:43.782822 32518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc2be006faba40ff8464aab72672ea0f: Sending DeleteTablet for 3 replicas of tablet a54d2e87a8484fa1b6fa71352b945963
I20260810 06:46:43.783865 527 tablet_service.cc:1514] Processing DeleteTablet for tablet a54d2e87a8484fa1b6fa71352b945963 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:43 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:33374
I20260810 06:46:43.783892 32746 tablet_service.cc:1514] Processing DeleteTablet for tablet a54d2e87a8484fa1b6fa71352b945963 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:43 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:57998
I20260810 06:46:43.783952 755 tablet_service.cc:1514] Processing DeleteTablet for tablet a54d2e87a8484fa1b6fa71352b945963 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:43 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:59660
I20260810 06:46:43.794016 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 32577
I20260810 06:46:43.799713 949 tablet_replica.cc:330] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: stopping tablet replica
I20260810 06:46:43.800181 949 raft_consensus.cc:2229] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:46:43.800684 949 raft_consensus.cc:2258] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:43.802021 948 tablet_replica.cc:330] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: stopping tablet replica
I20260810 06:46:43.802459 948 raft_consensus.cc:2229] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:46:43.803200 948 raft_consensus.cc:2258] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:43.804168 947 tablet_replica.cc:330] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: stopping tablet replica
I20260810 06:46:43.804561 947 raft_consensus.cc:2229] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:46:43.805049 947 raft_consensus.cc:2258] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:43.805554 949 ts_tablet_manager.cc:1905] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:43.807096 948 ts_tablet_manager.cc:1905] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:43.807322 947 ts_tablet_manager.cc:1905] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:43.812938 949 ts_tablet_manager.cc:1918] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:43.813174 949 log.cc:1196] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/a54d2e87a8484fa1b6fa71352b945963
I20260810 06:46:43.813822 949 ts_tablet_manager.cc:1939] T a54d2e87a8484fa1b6fa71352b945963 P f9a5e7c69e554f6594b6b387d99c8419: Deleting consensus metadata
I20260810 06:46:43.814169 947 ts_tablet_manager.cc:1918] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:43.814430 947 log.cc:1196] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/a54d2e87a8484fa1b6fa71352b945963
I20260810 06:46:43.814479 916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.794987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.10.242.1:44715)
0810 06:46:43.795572 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.795578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.795625 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:46:43.795689 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.795694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.795774 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.795864 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:43.798071 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.799361 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.801373 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.801988 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.802112 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.805906 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:43.805919 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:43.805934 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:43.805974 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:43.809862 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.810853 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.810867 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.810870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.811068 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.811630 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.811635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.811639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.812111 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:43.812435 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.813768 (+ 1333us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.814036 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431}
I20260810 06:46:43.815165 947 ts_tablet_manager.cc:1939] T a54d2e87a8484fa1b6fa71352b945963 P 41e8864b8124421590155dff124dfddc: Deleting consensus metadata
I20260810 06:46:43.815310 384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:43.815927 32486 catalog_manager.cc:4841] TS f9a5e7c69e554f6594b6b387d99c8419 (127.10.242.1:44715): tablet a54d2e87a8484fa1b6fa71352b945963 (table test-table [id=9d2a28bf701548208bb9e549ce77953e]) successfully deleted
I20260810 06:46:43.816566 948 ts_tablet_manager.cc:1918] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:43.816808 948 log.cc:1196] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/a54d2e87a8484fa1b6fa71352b945963
I20260810 06:46:43.817082 32486 catalog_manager.cc:4841] TS 41e8864b8124421590155dff124dfddc (127.10.242.2:35873): tablet a54d2e87a8484fa1b6fa71352b945963 (table test-table [id=9d2a28bf701548208bb9e549ce77953e]) successfully deleted
I20260810 06:46:43.817448 948 ts_tablet_manager.cc:1939] T a54d2e87a8484fa1b6fa71352b945963 P acb39e30cc9f4e1788aefd52b0a873c9: Deleting consensus metadata
I20260810 06:46:43.819234 32493 catalog_manager.cc:4841] TS acb39e30cc9f4e1788aefd52b0a873c9 (127.10.242.3:46651): tablet a54d2e87a8484fa1b6fa71352b945963 (table test-table [id=9d2a28bf701548208bb9e549ce77953e]) successfully deleted
I20260810 06:46:43.905342 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 32577
I20260810 06:46:43.931428 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 401
I20260810 06:46:43.946434 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:43.932265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.10.242.2:35873)
0810 06:46:43.932441 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:43.932446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:43.932513 (+ 67us) server_negotiation.cc:407] Connection header received
0810 06:46:43.932713 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:43.932717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:43.932797 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:43.932887 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:43.934721 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.935522 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.937411 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:43.937877 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:43.937995 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:43.940749 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:43.940764 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:43.940782 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:43.940821 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:43.943617 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.944264 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.944272 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.944274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.944412 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:43.944787 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:43.944790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:43.944792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:43.945177 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:43.945444 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:43.945776 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 06:46:43.946013 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:46:43.947223 628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:44.066223 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 401
I20260810 06:46:44.090273 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 642
I20260810 06:46:44.105435 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:44.091074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.10.242.3:46651)
0810 06:46:44.091282 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:44.091288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:44.091307 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:46:44.091472 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:44.091477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:44.091569 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:44.091657 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:44.092966 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:44.093925 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:44.095438 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:44.095870 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:44.095996 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:44.098578 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:44.098598 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:44.098622 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:44.098676 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:44.102626 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:44.103296 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:44.103306 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:44.103310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:44.103501 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:44.103823 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:44.103829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:44.103832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:44.104311 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:44.104639 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:44.104813 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:46:44.105038 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:46:44.106326 855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:44.227356 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 642
I20260810 06:46:44.250995 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 32453
I20260810 06:46:44.252213 32567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:44.362440 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 32453
I20260810 06:46:44.391551 11208 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:46:44.735749 11208 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:46:44.735948 11208 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:46:45.080052 11208 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:46:45.080 UTC [30907] LOG: received smart shutdown request
2026-08-10 06:46:45.082 UTC [30907] LOG: background worker "logical replication launcher" (PID 30923) exited with exit code 1
2026-08-10 06:46:45.082 UTC [30914] LOG: shutting down
2026-08-10 06:46:45.213 UTC [30907] LOG: database system is shut down
2026-08-10T06:46:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50439 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:46:45 dist-test-slave-zj8q krb5kdc[970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:46:45 dist-test-slave-zj8q krb5kdc[970](info): set up 2 sockets
Aug 10 06:46:45 dist-test-slave-zj8q krb5kdc[970](info): commencing operation
krb5kdc: starting...
W20260810 06:46:46.557463 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:46:46 dist-test-slave-zj8q krb5kdc[970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:46:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:46:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:46:46.803439 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:41971
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:41971,127.10.242.61:38019,127.10.242.60:43295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:47.143103 989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:47.143409 989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:47.143527 989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:47.150859 989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:47.150969 989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:47.151027 989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:47.151118 989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:47.151186 989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:47.161146 989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:41971,127.10.242.61:38019,127.10.242.60:43295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:41971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:47.162519 989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:47.163863 989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:47.175599 995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:47.175570 1000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:47.175844 996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:47.176357 989 server_base.cc:1034] running on GCE node
I20260810 06:46:47.177121 989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:47.178483 989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:47.179665 989 hybrid_clock.cc:648] HybridClock initialized: now 1786344407179615 us; error 44 us; skew 500 ppm
I20260810 06:46:47.179986 989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:47 dist-test-slave-zj8q krb5kdc[970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344407, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:47.187608 989 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:46:47.189095 989 webserver.cc:460] Webserver started at http://127.10.242.62:43535/ using document root <none> and password file <none>
I20260810 06:46:47.189786 989 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:47.189896 989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:47.190244 989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:47.193194 989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "4806ce73b8e842cf9f9a16d100e2c0c5"
format_stamp: "Formatted at 2026-08-10 06:46:47 on dist-test-slave-zj8q"
server_key: "a24272c125a2a25f8c92a4c43a136176"
server_key_iv: "3426f211fda08e8f3da742f129bf4547"
server_key_version: "encryptionkey@0"
I20260810 06:46:47.193985 989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "4806ce73b8e842cf9f9a16d100e2c0c5"
format_stamp: "Formatted at 2026-08-10 06:46:47 on dist-test-slave-zj8q"
server_key: "a24272c125a2a25f8c92a4c43a136176"
server_key_iv: "3426f211fda08e8f3da742f129bf4547"
server_key_version: "encryptionkey@0"
I20260810 06:46:47.198969 989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 06:46:47.202417 1012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:47.203727 989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:46:47.203890 989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "4806ce73b8e842cf9f9a16d100e2c0c5"
format_stamp: "Formatted at 2026-08-10 06:46:47 on dist-test-slave-zj8q"
server_key: "a24272c125a2a25f8c92a4c43a136176"
server_key_iv: "3426f211fda08e8f3da742f129bf4547"
server_key_version: "encryptionkey@0"
I20260810 06:46:47.204073 989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:47.230400 989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:47.234125 989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:47.234433 989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:47.251958 989 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:41971
I20260810 06:46:47.251992 1091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:41971 every 8 connection(s)
I20260810 06:46:47.253273 989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:46:47.256786 1092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:47.258493 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 989
I20260810 06:46:47.258767 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:46:47.259419 11208 external_mini_cluster.cc:1428] Setting key 886858eb0f888875a6b88eee10394b5c
I20260810 06:46:47.263368 1092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:46:47 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344407, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.61@KRBTEST.COM" created.
I20260810 06:46:47.289837 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.266519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41971 (local address 127.0.0.1:42604)
0810 06:46:47.271451 (+ 4932us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.271488 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:47.273192 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:47.273977 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:47.273983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:47.274712 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:47.275047 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:47.275064 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.276616 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.276620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:47.277722 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:47.277733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.278717 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.278725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:47.278884 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:47.279532 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:47.279554 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:47.282007 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:47.285269 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:47.285287 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:47.285300 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:47.285863 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:47.286323 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:47.286328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:47.286332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:47.286494 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:47.287321 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:47.287328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:47.287592 (+ 264us) client_negotiation.cc:769] Sending connection context
0810 06:46:47.287830 (+ 238us) client_negotiation.cc:240] Negotiation successful
0810 06:46:47.288344 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4452,"mutex_wait_us":1,"thread_start_us":3913,"threads_started":1}
I20260810 06:46:47.289855 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.266519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.10.242.62:41971)
0810 06:46:47.271313 (+ 4794us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:47.271323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:47.271605 (+ 282us) server_negotiation.cc:407] Connection header received
0810 06:46:47.273539 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:47.273544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:47.273759 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:47.273930 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:47.275240 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.276458 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.277859 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.278472 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.278659 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:47.282251 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:47.282260 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:47.282273 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:47.282303 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:47.285023 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:47.286005 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:47.286013 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:47.286015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:47.286222 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:47.286605 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:47.286610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:47.286614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:47.287178 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:47.287360 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:47.287976 (+ 616us) server_negotiation.cc:299] Negotiation successful
0810 06:46:47.288456 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":213,"server-negotiator.queue_time_us":4636,"thread_start_us":4303,"threads_started":1}
I20260810 06:46:47.293058 1092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:46:47.294139 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.293749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32900)
0810 06:46:47.293916 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.294021 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 06:46:47.294431 1018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
I20260810 06:46:47.295950 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.295605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32912)
0810 06:46:47.295755 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.295818 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 06:46:47.296164 1092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38019: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab.
Entry for principal kudu/127.10.242.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab.
I20260810 06:46:47.316902 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.61:38019
--webserver_interface=127.10.242.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab
--principal=kudu/127.10.242.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:41971,127.10.242.61:38019,127.10.242.60:43295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:46:47.345580 1092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } attempt: 1
I20260810 06:46:47.346735 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.346273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32916)
0810 06:46:47.346466 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.346597 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 06:46:47.347774 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.347398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32928)
0810 06:46:47.347579 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.347645 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:46:47.347961 1092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38019: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
I20260810 06:46:47.416182 1092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } attempt: 2
I20260810 06:46:47.417289 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.416823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32938)
0810 06:46:47.417056 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.417150 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:46:47.418340 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.417968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32952)
0810 06:46:47.418102 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.418210 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:46:47.418540 1092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38019: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
I20260810 06:46:47.509749 1092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } attempt: 3
I20260810 06:46:47.510991 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.510542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32964)
0810 06:46:47.510753 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.510861 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 06:46:47.512015 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.511617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32976)
0810 06:46:47.511796 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.511871 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260810 06:46:47.512231 1092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38019: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
W20260810 06:46:47.642802 1107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:47.643064 1107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:47.643198 1107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:47.650261 1107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:47.650367 1107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:47.650444 1107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:47.650516 1107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:47.650574 1107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:47.655455 1092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } attempt: 4
I20260810 06:46:47.656646 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.656100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32984)
0810 06:46:47.656423 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.656513 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260810 06:46:47.657663 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.657317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32992)
0810 06:46:47.657442 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.657540 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:46:47.657874 1092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38019: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38019: connect: Connection refused (error 111)
I20260810 06:46:47.660575 1107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:41971,127.10.242.61:38019,127.10.242.60:43295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab
--principal=kudu/127.10.242.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.61:38019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.242.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:47.661821 1107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:47.663136 1107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:47.674404 1118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:47.674428 1120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:47.674470 1124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:47.675172 1107 server_base.cc:1034] running on GCE node
I20260810 06:46:47.675987 1107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:47.677508 1107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:47.678711 1107 hybrid_clock.cc:648] HybridClock initialized: now 1786344407678666 us; error 34 us; skew 500 ppm
I20260810 06:46:47.679036 1107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:47 dist-test-slave-zj8q krb5kdc[970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344407, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:47.682057 1107 init.cc:377] Logged in from keytab as kudu/127.10.242.61@KRBTEST.COM (short username kudu)
I20260810 06:46:47.683527 1107 webserver.cc:460] Webserver started at http://127.10.242.61:40777/ using document root <none> and password file <none>
I20260810 06:46:47.684177 1107 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:47.684293 1107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:47.684582 1107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:47.687502 1107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/data/instance:
uuid: "fd732f92a97f45579e05a46446a3794a"
format_stamp: "Formatted at 2026-08-10 06:46:47 on dist-test-slave-zj8q"
server_key: "3080afe12e9aa60c0da88b80724f8622"
server_key_iv: "57302e9162ece050203b4107831385b4"
server_key_version: "encryptionkey@0"
I20260810 06:46:47.688294 1107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/wal/instance:
uuid: "fd732f92a97f45579e05a46446a3794a"
format_stamp: "Formatted at 2026-08-10 06:46:47 on dist-test-slave-zj8q"
server_key: "3080afe12e9aa60c0da88b80724f8622"
server_key_iv: "57302e9162ece050203b4107831385b4"
server_key_version: "encryptionkey@0"
I20260810 06:46:47.693190 1107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:46:47.696406 1132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:47.697620 1107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:46:47.697782 1107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/wal
uuid: "fd732f92a97f45579e05a46446a3794a"
format_stamp: "Formatted at 2026-08-10 06:46:47 on dist-test-slave-zj8q"
server_key: "3080afe12e9aa60c0da88b80724f8622"
server_key_iv: "57302e9162ece050203b4107831385b4"
server_key_version: "encryptionkey@0"
I20260810 06:46:47.697947 1107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:47.716058 1107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:47.719797 1107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:47.720050 1107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:47.737267 1107 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.61:38019
I20260810 06:46:47.737303 1227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.61:38019 every 8 connection(s)
I20260810 06:46:47.738704 1107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
I20260810 06:46:47.740976 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 1107
I20260810 06:46:47.741173 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-1/wal/instance
I20260810 06:46:47.741724 11208 external_mini_cluster.cc:1428] Setting key 1aaa85cb04b08c262782a1aa5865ac08
I20260810 06:46:47.742795 1229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:47.751284 1229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:46:47 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344407, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.242.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.60@KRBTEST.COM" created.
I20260810 06:46:47.771914 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.754296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42610 (local address 127.10.242.62:41971)
0810 06:46:47.754510 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:47.754516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:47.755559 (+ 1043us) server_negotiation.cc:407] Connection header received
0810 06:46:47.757394 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:47.757400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:47.757480 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:47.757591 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:47.759065 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.760047 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.761529 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.762206 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.762322 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:47.765772 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:47.765778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:47.765784 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:47.765826 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:47.768837 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:47.769562 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:47.769566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:47.769568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:47.769689 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:47.770101 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:47.770103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:47.770105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:47.770468 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:47.770629 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:47.771380 (+ 751us) server_negotiation.cc:299] Negotiation successful
0810 06:46:47.771598 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:46:47.772737 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.754626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41971 (local address 127.0.0.1:42610)
0810 06:46:47.755384 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.755435 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:47.757117 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:47.757780 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:47.757787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:47.758347 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:47.758878 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:47.758895 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.760196 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.760200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:47.761387 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:47.761399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.762387 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.762392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:47.762571 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:47.763196 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:47.763219 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:47.765531 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:47.769008 (+ 3477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:47.769017 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:47.769031 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:47.769464 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:47.769767 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:47.769770 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:47.769772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:47.770022 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:47.770579 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:47.770585 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:47.771006 (+ 421us) client_negotiation.cc:769] Sending connection context
0810 06:46:47.771256 (+ 250us) client_negotiation.cc:240] Negotiation successful
0810 06:46:47.771694 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":310,"threads_started":1}
I20260810 06:46:47.774346 1229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:46:47 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344407, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
I20260810 06:46:47.788921 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.775036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:32994)
0810 06:46:47.775222 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.775240 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:47.775344 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:47.776474 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:47.776479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:47.776910 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:47.777080 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:47.777092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.778813 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.778817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:47.779651 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:47.779661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.780531 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.780536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:47.780724 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:47.781248 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:47.781270 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:47.783580 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:47.786516 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:47.786521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:47.786525 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:47.786955 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:47.787287 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:47.787291 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:47.787294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:47.787472 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:47.788057 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:47.788061 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:47.788201 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 06:46:47.788336 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 06:46:47.788591 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 06:46:47.789026 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.775291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.10.242.61:38019)
0810 06:46:47.776000 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:47.776007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:47.776038 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:46:47.776091 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:47.776098 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:47.776311 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:47.776454 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:47.777242 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.778663 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.779787 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.780369 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.780468 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:47.783773 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:47.783784 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:47.783796 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:47.783828 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:47.786389 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:47.787065 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:47.787072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:47.787074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:47.787187 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:47.787576 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:47.787579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:47.787581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:47.787931 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:47.788087 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:47.788445 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 06:46:47.788720 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":563,"thread_start_us":426,"threads_started":1}
I20260810 06:46:47.791261 1229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:46:47.792289 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.791878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57374)
0810 06:46:47.792043 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.792153 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 06:46:47.792541 1137 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
I20260810 06:46:47.793987 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.793695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57376)
0810 06:46:47.793823 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.793897 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 06:46:47.794220 1229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:43295: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab.
Entry for principal kudu/127.10.242.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab.
I20260810 06:46:47.800926 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.60:43295
--webserver_interface=127.10.242.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab
--principal=kudu/127.10.242.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:37601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:41971,127.10.242.61:38019,127.10.242.60:43295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:46:47.843526 1229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } attempt: 1
I20260810 06:46:47.844676 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.844235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57378)
0810 06:46:47.844418 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.844521 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 06:46:47.845651 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.845296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57386)
0810 06:46:47.845452 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.845534 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:46:47.845846 1229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:43295: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
I20260810 06:46:47.914043 1229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } attempt: 2
I20260810 06:46:47.915159 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.914714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57398)
0810 06:46:47.914905 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.914995 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:46:47.916070 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.915751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57410)
0810 06:46:47.915893 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.915960 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:46:47.916268 1229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:43295: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
I20260810 06:46:47.957114 1092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } attempt: 5
Aug 10 06:46:47 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344407, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
I20260810 06:46:47.969872 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.957869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:33004)
0810 06:46:47.958086 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.958106 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:47.958278 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:47.958660 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:47.958665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:47.959048 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:47.959234 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:47.959244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.960240 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.960246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:47.961122 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:47.961134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.961877 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.961883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:47.962065 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:47.962711 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:47.962729 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:47.964733 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:47.967588 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:47.967594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:47.967598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:47.967946 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:47.968295 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:47.968300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:47.968304 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:47.968435 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:47.968934 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:47.968939 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:47.969083 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 06:46:47.969228 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 06:46:47.969497 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 06:46:47.969872 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.957917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.10.242.61:38019)
0810 06:46:47.958089 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:47.958095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:47.958182 (+ 87us) server_negotiation.cc:407] Connection header received
0810 06:46:47.958430 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:47.958434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:47.958501 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:47.958586 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:47.959407 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.960093 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.961272 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:47.961730 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:47.961853 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:47.964917 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:47.964922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:47.964925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:47.964958 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:47.967457 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:47.968057 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:47.968062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:47.968064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:47.968189 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:47.968536 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:47.968541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:47.968544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:47.968819 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:47.968951 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:47.969354 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 06:46:47.969555 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:46:47.971119 1092 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:46:47.972219 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.971779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57420)
0810 06:46:47.972022 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.972090 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260810 06:46:47.973045 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:47.972735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57430)
0810 06:46:47.972882 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:47.972952 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 06:46:47.973235 1092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:43295: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
I20260810 06:46:48.007457 1229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } attempt: 3
I20260810 06:46:48.008599 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.008060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57440)
0810 06:46:48.008304 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.008427 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 06:46:48.009641 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.009220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57454)
0810 06:46:48.009448 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.009526 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260810 06:46:48.009859 1229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:43295: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
I20260810 06:46:48.024441 1092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } attempt: 1
I20260810 06:46:48.025486 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.025042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57470)
0810 06:46:48.025226 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.025325 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:46:48.026439 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.026102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57482)
0810 06:46:48.026261 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.026334 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 06:46:48.026674 1092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:43295: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
I20260810 06:46:48.094887 1092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } attempt: 2
I20260810 06:46:48.096156 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.095523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57494)
0810 06:46:48.095795 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.095924 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 06:46:48.097218 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.096821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57506)
0810 06:46:48.096977 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.097078 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 06:46:48.097431 1092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:43295: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
W20260810 06:46:48.130756 1243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:48.131045 1243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:48.131165 1243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:48.138423 1243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:48.138542 1243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:48.138617 1243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:48.138693 1243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:48.138754 1243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:48.148828 1243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:37601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:41971,127.10.242.61:38019,127.10.242.60:43295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab
--principal=kudu/127.10.242.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.60:43295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.242.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:48.150236 1243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:48.151574 1243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:46:48.153085 1229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } attempt: 4
I20260810 06:46:48.154258 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.153729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57520)
0810 06:46:48.153959 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.154047 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 06:46:48.155273 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.154964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57532)
0810 06:46:48.155102 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.155172 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:46:48.155493 1229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:43295: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
W20260810 06:46:48.163470 1253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:48.163525 1256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:48.163527 1261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:48.164542 1243 server_base.cc:1034] running on GCE node
I20260810 06:46:48.165266 1243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:48.166623 1243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:48.167807 1243 hybrid_clock.cc:648] HybridClock initialized: now 1786344408167755 us; error 43 us; skew 500 ppm
I20260810 06:46:48.168124 1243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:48 dist-test-slave-zj8q krb5kdc[970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344408, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:48.171113 1243 init.cc:377] Logged in from keytab as kudu/127.10.242.60@KRBTEST.COM (short username kudu)
I20260810 06:46:48.172580 1243 webserver.cc:460] Webserver started at http://127.10.242.60:36645/ using document root <none> and password file <none>
I20260810 06:46:48.173221 1243 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:48.173329 1243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:48.173622 1243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:48.176518 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/data/instance:
uuid: "446eda06b04d47bca3cc4dbfe5860fa3"
format_stamp: "Formatted at 2026-08-10 06:46:48 on dist-test-slave-zj8q"
server_key: "a202036118f5703b8f8604829159fb25"
server_key_iv: "0911bf936f6c14c5e1bfeebda937e985"
server_key_version: "encryptionkey@0"
I20260810 06:46:48.177429 1243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/wal/instance:
uuid: "446eda06b04d47bca3cc4dbfe5860fa3"
format_stamp: "Formatted at 2026-08-10 06:46:48 on dist-test-slave-zj8q"
server_key: "a202036118f5703b8f8604829159fb25"
server_key_iv: "0911bf936f6c14c5e1bfeebda937e985"
server_key_version: "encryptionkey@0"
I20260810 06:46:48.182410 1243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260810 06:46:48.185659 1271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:48.186896 1243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:46:48.187057 1243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/wal
uuid: "446eda06b04d47bca3cc4dbfe5860fa3"
format_stamp: "Formatted at 2026-08-10 06:46:48 on dist-test-slave-zj8q"
server_key: "a202036118f5703b8f8604829159fb25"
server_key_iv: "0911bf936f6c14c5e1bfeebda937e985"
server_key_version: "encryptionkey@0"
I20260810 06:46:48.187237 1243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:48.200623 1243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:48.203665 1092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } attempt: 3
I20260810 06:46:48.204788 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.204261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57542)
0810 06:46:48.204472 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.204574 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 06:46:48.204886 1243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:48.205160 1243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:48.205871 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.205439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57554)
0810 06:46:48.205640 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.205736 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260810 06:46:48.206108 1092 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:43295: Network error: Client connection negotiation failed: client connection to 127.10.242.60:43295: connect: Connection refused (error 111)
I20260810 06:46:48.222546 1243 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.60:43295
I20260810 06:46:48.222585 1342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.60:43295 every 8 connection(s)
I20260810 06:46:48.223825 1243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
I20260810 06:46:48.224395 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 1243
I20260810 06:46:48.224582 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344206489568-11208-0/minicluster-data/master-2/wal/instance
I20260810 06:46:48.225164 11208 external_mini_cluster.cc:1428] Setting key 8828294b32df5a11a5ac2ea8bb73d10f
I20260810 06:46:48.227707 1343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:48.236404 1343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:46:48 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
Aug 10 06:46:48 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344408, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:48.249629 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.227096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57556 (local address 127.10.242.60:43295)
0810 06:46:48.227970 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.227989 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.228331 (+ 342us) server_negotiation.cc:407] Connection header received
0810 06:46:48.230507 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.230536 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.231017 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.231446 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.232843 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.234835 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.236245 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.237269 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.237448 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.242025 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.242032 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.242042 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.242075 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.245232 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.246189 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.246198 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.246206 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.246412 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.246815 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.246820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.246823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.247487 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.247730 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.248253 (+ 523us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.248562 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":370,"threads_started":1}
I20260810 06:46:48.255285 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.239109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:41971 (local address 127.0.0.1:42612)
0810 06:46:48.239903 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.239949 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:48.240110 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:48.240467 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:48.240474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:48.241159 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:48.241533 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.241547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.242596 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.242600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.244192 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.244214 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.245165 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.245172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.245347 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.246038 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:48.246057 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:48.248825 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:48.252816 (+ 3991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.252826 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.252830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.253238 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.253624 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.253627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.253629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.253758 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.254260 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:48.254267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:48.254427 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 06:46:48.254571 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 06:46:48.254875 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":387,"threads_started":1}
I20260810 06:46:48.255309 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.239179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42612 (local address 127.10.242.62:41971)
0810 06:46:48.239385 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.239391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.240041 (+ 650us) server_negotiation.cc:407] Connection header received
0810 06:46:48.240229 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.240233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.240293 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.240381 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.241706 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.242451 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.244352 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.244975 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.245091 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.249056 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.249068 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.249071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.249107 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.252666 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.253353 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.253359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.253363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.253508 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.253861 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.253866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.253869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.254116 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.254276 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.254733 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.254941 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:46:48.256611 1343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:46:48 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344408, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
Aug 10 06:46:48 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:48.270499 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.257407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38019 (local address 127.0.0.1:33020)
0810 06:46:48.257659 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.257673 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:48.257772 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:48.258081 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:48.258086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:48.258491 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:48.258683 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.258695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.259672 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.259676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.260715 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.260730 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.261465 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.261469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.261625 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.262201 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:48.262222 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:48.264409 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:48.267882 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.267887 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.267890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.268397 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.268803 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.268807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.268811 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.268992 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.269667 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:48.269672 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:48.269826 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 06:46:48.269943 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:46:48.270138 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260810 06:46:48.270509 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.257389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33020 (local address 127.10.242.61:38019)
0810 06:46:48.257550 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.257555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.257746 (+ 191us) server_negotiation.cc:407] Connection header received
0810 06:46:48.257883 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.257887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.257951 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.258031 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.258850 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.259519 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.260859 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.261323 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.261450 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.264583 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.264588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.264591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.264623 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.267743 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.268511 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.268516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.268520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.268698 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.269180 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.269184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.269186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.269504 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.269690 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.270038 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.270252 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:46:48.271749 1343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:46:48.274284 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.258298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.10.242.62:41971)
0810 06:46:48.258464 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.258470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.258486 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:46:48.258532 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.258537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.258613 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.258712 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.260288 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.261194 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.262680 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.263263 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.263379 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.267924 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.267930 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.267935 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.267971 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.271669 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.272381 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.272386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.272389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.272556 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.272965 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.272969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.272971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.273232 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.273488 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.273771 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.273976 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 06:46:48 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344408, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
Aug 10 06:46:48 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
I20260810 06:46:48.284466 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.272423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57568)
0810 06:46:48.272552 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.272566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:48.272656 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:48.273085 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:48.273089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:48.273432 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:48.273564 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.273573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.274647 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.274652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.275460 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.275471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.276356 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.276361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.276503 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.277005 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:48.277020 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:48.279207 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:48.282096 (+ 2889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.282101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.282105 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.282603 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.282946 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.282951 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.282955 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.283129 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.283587 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:48.283592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:48.283752 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 06:46:48.283890 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 06:46:48.284117 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 06:46:48.284705 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.272732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57568 (local address 127.10.242.60:43295)
0810 06:46:48.272863 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.272867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.272881 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:46:48.272916 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.272920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.272968 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.273068 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.273688 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.274534 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.275624 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.276195 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.276313 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.279376 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.279380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.279383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.279411 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.281952 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.282714 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.282718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.282720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.282836 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.283228 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.283231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.283233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.283474 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.283575 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.283994 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.284316 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":47}
I20260810 06:46:48.290568 1343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3: Bootstrap starting.
I20260810 06:46:48.291153 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.275713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.10.242.61:38019)
0810 06:46:48.275863 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.275868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.275907 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:46:48.276075 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.276079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.276143 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.276207 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.277531 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.278476 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.279966 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.280578 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.280697 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.285377 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.285382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.285385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.285414 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.288467 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.289129 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.289134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.289137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.289294 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.289713 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.289718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.289721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.290055 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.290364 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.290635 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.290851 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:46:48.294404 1343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:48.295570 1343 log.cc:824] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:48.296715 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:46:48.298293 1343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3: No bootstrap required, opened a new log
I20260810 06:46:48.308367 1343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } }
I20260810 06:46:48.308933 1343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:48.309154 1343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 446eda06b04d47bca3cc4dbfe5860fa3, State: Initialized, Role: FOLLOWER
I20260810 06:46:48.309983 1343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } }
I20260810 06:46:48.317212 1362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.317642 1362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:46:48.319761 1343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:48.332417 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.302076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.10.242.62:41971)
0810 06:46:48.302724 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.302731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.312485 (+ 9754us) server_negotiation.cc:407] Connection header received
0810 06:46:48.313108 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.313116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.313213 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.313289 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.316560 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.317667 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.319994 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.320638 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.320732 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.324928 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.324935 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.324941 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.324987 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.328537 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.329646 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.329653 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.329657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.329842 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.330417 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.330422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.330426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.330822 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.331069 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.331748 (+ 679us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.332006 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502}
I20260810 06:46:48.333694 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.302310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57570 (local address 127.10.242.60:43295)
0810 06:46:48.303769 (+ 1459us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.303775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.316374 (+ 12599us) server_negotiation.cc:407] Connection header received
0810 06:46:48.316506 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.316512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.316588 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.316686 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.319183 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.320198 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.322232 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.322863 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.322957 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.326652 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.326658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.326663 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.326703 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.330403 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.331398 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.331403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.331406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.331577 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.331987 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.331992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.331995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.332336 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.332558 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.333134 (+ 576us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.333350 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1301}
I20260810 06:46:48.338724 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.302503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33042 (local address 127.10.242.61:38019)
0810 06:46:48.307610 (+ 5107us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.307617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.322117 (+ 14500us) server_negotiation.cc:407] Connection header received
0810 06:46:48.322355 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.322361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.322441 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.322506 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.324570 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.325675 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.327567 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.328213 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.328307 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.332005 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.332011 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.332015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.332052 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.335877 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.336551 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.336557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.336562 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.336733 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.337160 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.337166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.337169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.337567 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.337819 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.338102 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.338371 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4977}
I20260810 06:46:48.341966 1362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:46:48.342263 1362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } }
I20260810 06:46:48.344959 1362 leader_election.cc:288] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4806ce73b8e842cf9f9a16d100e2c0c5 (127.10.242.62:41971), fd732f92a97f45579e05a46446a3794a (127.10.242.61:38019)
W20260810 06:46:48.351365 1386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:46:48.353451 1386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:46:48.345680 1243 master_runner.cc:186] Error getting master registration for 127.10.242.62:41971: OK, instance_id { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" instance_seqno: 1786344407246859 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:46:48.356698 1243 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:46:48.383334 1092 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } attempt: 4
Aug 10 06:46:48 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344407, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
I20260810 06:46:48.399924 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.383896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57572)
0810 06:46:48.384110 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.384132 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:48.384242 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:48.384578 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:48.384583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:48.385106 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:48.385322 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.385335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.386867 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.386873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.388362 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.388378 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.389581 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.389586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.389804 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.390624 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:48.390659 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:48.393507 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:48.397285 (+ 3778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.397290 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.397294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.397769 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.398140 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.398146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.398185 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.398372 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.398940 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:48.398945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:48.399127 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 06:46:48.399280 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 06:46:48.399542 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 06:46:48.399963 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.384083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.10.242.60:43295)
0810 06:46:48.384244 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.384249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.384266 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:46:48.384355 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.384358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.384431 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.384520 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.385472 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.386738 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.388508 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.389231 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.389358 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.393703 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.393710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.393714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.393751 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.397165 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.397875 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.397879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.397881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.398018 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.398479 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.398484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.398488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.398819 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.398955 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.399406 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.399607 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:46:48.404907 1092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5: Bootstrap starting.
I20260810 06:46:48.408088 1092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:48.409147 1092 log.cc:824] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:48.412614 1092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5: No bootstrap required, opened a new log
I20260810 06:46:48.423271 1092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } }
I20260810 06:46:48.423672 1092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:48.423808 1092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4806ce73b8e842cf9f9a16d100e2c0c5, State: Initialized, Role: FOLLOWER
I20260810 06:46:48.424540 1092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } }
I20260810 06:46:48.425684 1392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.426136 1392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:46:48.426940 1092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:48.426966 1056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" is_pre_election: true
I20260810 06:46:48.427552 1056 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 446eda06b04d47bca3cc4dbfe5860fa3 in term 0.
I20260810 06:46:48.428220 1272 leader_election.cc:302] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 446eda06b04d47bca3cc4dbfe5860fa3, 4806ce73b8e842cf9f9a16d100e2c0c5; no voters:
I20260810 06:46:48.428671 1362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:46:48.428805 1362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:46:48.428910 1362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:48.431996 1362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } }
I20260810 06:46:48.432678 1362 leader_election.cc:288] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [CANDIDATE]: Term 1 election: Requested vote from peers 4806ce73b8e842cf9f9a16d100e2c0c5 (127.10.242.62:41971), fd732f92a97f45579e05a46446a3794a (127.10.242.61:38019)
I20260810 06:46:48.433235 1056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5"
I20260810 06:46:48.433444 1056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:48.437538 1056 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 446eda06b04d47bca3cc4dbfe5860fa3 in term 1.
I20260810 06:46:48.438024 1272 leader_election.cc:302] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 446eda06b04d47bca3cc4dbfe5860fa3, 4806ce73b8e842cf9f9a16d100e2c0c5; no voters:
I20260810 06:46:48.438385 1362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:48.442109 1362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 446eda06b04d47bca3cc4dbfe5860fa3, State: Running, Role: LEADER
I20260810 06:46:48.442728 1362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } }
I20260810 06:46:48.446767 1405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 446eda06b04d47bca3cc4dbfe5860fa3. Latest consensus state: current_term: 1 leader_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.449386 1405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:48.454710 1229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } attempt: 5
W20260810 06:46:48.457310 1278 tablet.cc:2369] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:46:48 dist-test-slave-zj8q krb5kdc[970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344407, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
I20260810 06:46:48.449954 989 master_runner.cc:186] Error getting master registration for 127.10.242.61:38019: OK, instance_id { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" instance_seqno: 1786344407732341 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:46:48.465165 989 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:46:48.470945 1414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:46:48.473512 1414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 06:46:48.476297 1406 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:46:48.476410 1406 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:46:48.477406 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.455263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:43295 (local address 127.0.0.1:57576)
0810 06:46:48.455486 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:48.455503 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:48.455622 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:48.455931 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:48.455936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:48.456453 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:48.456666 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.456678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.458092 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.458098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.459528 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:48.459544 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.460869 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.460875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:48.461085 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.461709 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:48.461732 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:48.464486 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:48.473718 (+ 9232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.473724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.473729 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.474279 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.475029 (+ 750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:48.475034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:48.475037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:48.475206 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:48.476408 (+ 1202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:48.476414 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:48.476580 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 06:46:48.476732 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 06:46:48.477038 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260810 06:46:48.478703 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:48.455312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57576 (local address 127.10.242.60:43295)
0810 06:46:48.455479 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:48.455485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:48.455556 (+ 71us) server_negotiation.cc:407] Connection header received
0810 06:46:48.455743 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:48.455749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:48.455822 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:48.455902 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:48.456799 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.457940 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.459756 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:48.460492 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:48.460615 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:48.464661 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:48.464668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:48.464673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:48.464710 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:48.473476 (+ 8766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.474519 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.474525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.474529 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.474820 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:48.475450 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:48.475456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:48.475459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:48.476199 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:48.477885 (+ 1686us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:48.478020 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:46:48.478306 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:46:48.484162 1229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a: Bootstrap starting.
W20260810 06:46:48.487123 1025 tablet.cc:2369] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:46:48.488510 1229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:48.490082 1056 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [term 1 FOLLOWER]: Refusing update from remote peer 446eda06b04d47bca3cc4dbfe5860fa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:48.490840 1229 log.cc:824] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:48.491272 1405 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:48.504424 1229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a: No bootstrap required, opened a new log
I20260810 06:46:48.517820 1392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 446eda06b04d47bca3cc4dbfe5860fa3. Latest consensus state: current_term: 1 leader_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.518187 1392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:46:48.522248 1229 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } }
I20260810 06:46:48.522919 1229 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:48.523137 1229 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd732f92a97f45579e05a46446a3794a, State: Initialized, Role: FOLLOWER
I20260810 06:46:48.524122 1229 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } }
I20260810 06:46:48.525115 1415 mvcc.cc:205] Tried to move back new op lower bound from 7316866697144606720 to 7316866696986898432. Current Snapshot: MvccSnapshot[applied={T|T < 7316866697144606720}]
I20260810 06:46:48.527630 1417 mvcc.cc:205] Tried to move back new op lower bound from 7316866697144606720 to 7316866696986898432. Current Snapshot: MvccSnapshot[applied={T|T < 7316866697144606720}]
I20260810 06:46:48.528584 1405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.528738 1392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.528864 1405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:48.528946 1392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:46:48.535485 1420 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.537675 1420 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:46:48.537635 1229 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:48.537613 1178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd732f92a97f45579e05a46446a3794a"
I20260810 06:46:48.538208 1178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:48.541746 1178 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 446eda06b04d47bca3cc4dbfe5860fa3 in term 1.
I20260810 06:46:48.544265 1414 catalog_manager.cc:1347] Generated new cluster ID: 077dca41df334bd69a38258c62834ba7
I20260810 06:46:48.544376 1414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:46:48.547045 1177 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [term 1 FOLLOWER]: Refusing update from remote peer 446eda06b04d47bca3cc4dbfe5860fa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:48.547899 1362 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:48.548712 1179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd732f92a97f45579e05a46446a3794a" is_pre_election: true
I20260810 06:46:48.549018 1179 raft_consensus.cc:2362] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 446eda06b04d47bca3cc4dbfe5860fa3 in term 1. Re-sending same reply.
I20260810 06:46:48.561949 1414 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:46:48.563778 1414 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:46:48.581473 1420 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 446eda06b04d47bca3cc4dbfe5860fa3. Latest consensus state: current_term: 1 leader_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.581512 1362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.581796 1420 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:46:48.581810 1362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:48.588773 1414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 446eda06b04d47bca3cc4dbfe5860fa3: Generated new TSK 0
I20260810 06:46:48.589823 1414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:46:48.584610 1436 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4806ce73b8e842cf9f9a16d100e2c0c5" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 41971 } } peers { permanent_uuid: "fd732f92a97f45579e05a46446a3794a" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38019 } } peers { permanent_uuid: "446eda06b04d47bca3cc4dbfe5860fa3" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 43295 } } }
I20260810 06:46:48.591192 1436 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:46:48.618942 1449 catalog_manager.cc:1259] Loaded cluster ID: 077dca41df334bd69a38258c62834ba7
I20260810 06:46:48.619128 1449 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a: loading cluster ID for follower catalog manager: success
I20260810 06:46:48.623188 1449 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a: acquiring CA information for follower catalog manager: success
I20260810 06:46:48.625828 1449 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fd732f92a97f45579e05a46446a3794a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:46:49.478474 1406 catalog_manager.cc:1259] Loaded cluster ID: 077dca41df334bd69a38258c62834ba7
I20260810 06:46:49.478579 1406 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5: loading cluster ID for follower catalog manager: success
I20260810 06:46:49.481009 1406 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5: acquiring CA information for follower catalog manager: success
I20260810 06:46:49.482776 1406 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4806ce73b8e842cf9f9a16d100e2c0c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:46:49.739651 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 989
I20260810 06:46:49.742105 1087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:49.851197 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 989
I20260810 06:46:49.879806 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 1107
I20260810 06:46:49.880867 1217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:49.985263 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 1107
I20260810 06:46:50.009994 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 1243
I20260810 06:46:50.011070 1335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:50.129482 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 1243
2026-08-10T06:46:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4807 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:46:50 dist-test-slave-zj8q krb5kdc[1471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:46:50 dist-test-slave-zj8q krb5kdc[1471](info): set up 2 sockets
Aug 10 06:46:50 dist-test-slave-zj8q krb5kdc[1471](info): commencing operation
krb5kdc: starting...
W20260810 06:46:51.358575 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:46:51 dist-test-slave-zj8q krb5kdc[1471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:46:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:46:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:46:51.598922 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:38269
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:51.920459 1490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:51.920708 1490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:51.920841 1490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:51.927713 1490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:51.927809 1490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:51.927881 1490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:51.927950 1490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:51.928007 1490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:51.937527 1490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:38269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:51.938784 1490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:51.940025 1490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:51.951071 1496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:51.951031 1508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:51.951104 1500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:51.952116 1490 server_base.cc:1034] running on GCE node
I20260810 06:46:51.952947 1490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:51.954263 1490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:51.955470 1490 hybrid_clock.cc:648] HybridClock initialized: now 1786344411955413 us; error 45 us; skew 500 ppm
I20260810 06:46:51.955796 1490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:51 dist-test-slave-zj8q krb5kdc[1471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344411, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:51.958837 1490 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:46:51.960209 1490 webserver.cc:460] Webserver started at http://127.10.242.62:38489/ using document root <none> and password file <none>
I20260810 06:46:51.960847 1490 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:51.960965 1490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:51.961257 1490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:51.964110 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "ee60ca306a964710af807f28ef95a78a"
format_stamp: "Formatted at 2026-08-10 06:46:51 on dist-test-slave-zj8q"
server_key: "c59f307951fcfaaf88caede2d23666f7"
server_key_iv: "44f5f852db5ad6aabb4cc197a247c6e0"
server_key_version: "encryptionkey@0"
I20260810 06:46:51.964841 1490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "ee60ca306a964710af807f28ef95a78a"
format_stamp: "Formatted at 2026-08-10 06:46:51 on dist-test-slave-zj8q"
server_key: "c59f307951fcfaaf88caede2d23666f7"
server_key_iv: "44f5f852db5ad6aabb4cc197a247c6e0"
server_key_version: "encryptionkey@0"
I20260810 06:46:51.969830 1490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:46:51.973377 1524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:51.974639 1490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 06:46:51.974802 1490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "ee60ca306a964710af807f28ef95a78a"
format_stamp: "Formatted at 2026-08-10 06:46:51 on dist-test-slave-zj8q"
server_key: "c59f307951fcfaaf88caede2d23666f7"
server_key_iv: "44f5f852db5ad6aabb4cc197a247c6e0"
server_key_version: "encryptionkey@0"
I20260810 06:46:51.974972 1490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:51.997411 1490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:52.001101 1490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:52.001358 1490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:52.020928 1490 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:38269
I20260810 06:46:52.020951 1602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:38269 every 8 connection(s)
I20260810 06:46:52.022243 1490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:46:52.025867 1603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:52.029491 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 1490
I20260810 06:46:52.029698 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:46:52.030387 11208 external_mini_cluster.cc:1428] Setting key efb51a537bd6d085a2e0c7c8f81c4cdd
I20260810 06:46:52.036664 1603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a: Bootstrap starting.
I20260810 06:46:52.040269 1603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:52.041415 1603 log.cc:824] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:52.044673 1603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a: No bootstrap required, opened a new log
Aug 10 06:46:52 dist-test-slave-zj8q krb5kdc[1471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:52.053781 1603 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee60ca306a964710af807f28ef95a78a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38269 } }
I20260810 06:46:52.054175 1603 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:52.054287 1603 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee60ca306a964710af807f28ef95a78a, State: Initialized, Role: FOLLOWER
I20260810 06:46:52.054390 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:52.032158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.10.242.62:38269)
0810 06:46:52.032946 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:52.032958 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:52.032999 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:46:52.035223 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:52.035246 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:52.035595 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:52.035921 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:52.037801 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:52.039169 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:52.040985 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:52.041698 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:52.041845 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:52.046835 (+ 4990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:52.046843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:52.046858 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:52.046904 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:52.050099 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:52.050970 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:52.050977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:52.050984 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:52.051173 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:52.051600 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:52.051604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:52.051606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:52.052188 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:52.052440 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:52.052928 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 06:46:52.053294 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":364,"threads_started":1}
I20260810 06:46:52.054896 1603 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee60ca306a964710af807f28ef95a78a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38269 } }
I20260810 06:46:52.055145 1603 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:46:52.055294 1603 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:46:52.055464 1603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:52.057893 1603 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee60ca306a964710af807f28ef95a78a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38269 } }
I20260810 06:46:52.058429 1603 leader_election.cc:302] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee60ca306a964710af807f28ef95a78a; no voters:
I20260810 06:46:52.058539 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:46:52.059085 1603 leader_election.cc:288] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:46:52.059190 1611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:52.061316 1603 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:52.065852 1611 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [term 1 LEADER]: Becoming Leader. State: Replica: ee60ca306a964710af807f28ef95a78a, State: Running, Role: LEADER
I20260810 06:46:52.066391 1611 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee60ca306a964710af807f28ef95a78a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38269 } }
I20260810 06:46:52.068848 1612 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee60ca306a964710af807f28ef95a78a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee60ca306a964710af807f28ef95a78a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38269 } } }
I20260810 06:46:52.069252 1612 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:52.069082 1613 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee60ca306a964710af807f28ef95a78a. Latest consensus state: current_term: 1 leader_uuid: "ee60ca306a964710af807f28ef95a78a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee60ca306a964710af807f28ef95a78a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38269 } } }
I20260810 06:46:52.069414 1613 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a [sys.catalog]: This master's current role is: LEADER
W20260810 06:46:52.074035 1632 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:46:52.074128 1632 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:46:52.074291 1635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:46:52.076411 1635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:46:52.085119 1635 catalog_manager.cc:1347] Generated new cluster ID: 5f8b97e41fc844eda6d6c344d3cb2e18
I20260810 06:46:52.085224 1635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:46:52.107378 1635 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:46:52.108726 1635 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:46:52.115602 1635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee60ca306a964710af807f28ef95a78a: Generated new TSK 0
I20260810 06:46:52.116484 1635 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:46:52.170821 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 1490
I20260810 06:46:52.171952 1597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:52.267858 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 1490
2026-08-10T06:46:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2134 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:46:52 dist-test-slave-zj8q krb5kdc[1647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:46:52 dist-test-slave-zj8q krb5kdc[1647](info): set up 2 sockets
Aug 10 06:46:52 dist-test-slave-zj8q krb5kdc[1647](info): commencing operation
krb5kdc: starting...
W20260810 06:46:53.491825 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:46:53 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:46:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:46:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:46:53.738648 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:44249
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:44249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:54.066882 1666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:54.067165 1666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:54.067304 1666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:54.074366 1666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:54.074465 1666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:54.074539 1666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:54.074612 1666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:54.074672 1666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:54.084192 1666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:44249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:44249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:54.085461 1666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:54.086814 1666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:54.098109 1674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:54.098059 1679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:54.098143 1676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:54.098956 1666 server_base.cc:1034] running on GCE node
I20260810 06:46:54.099676 1666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:54.100986 1666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:54.102227 1666 hybrid_clock.cc:648] HybridClock initialized: now 1786344414102177 us; error 30 us; skew 500 ppm
I20260810 06:46:54.102552 1666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:54 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:54.105823 1666 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:46:54.107409 1666 webserver.cc:460] Webserver started at http://127.10.242.62:40279/ using document root <none> and password file <none>
I20260810 06:46:54.108063 1666 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:54.108193 1666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:54.108503 1666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:54.111519 1666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "6d6866c2413b4331a25a7d5e368dfb33"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "0a4bde9a2fa89e8f1a1a086048e26155"
server_key_iv: "d0485eb1ab6eea4407bec8b5c5c93427"
server_key_version: "encryptionkey@0"
I20260810 06:46:54.112268 1666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "6d6866c2413b4331a25a7d5e368dfb33"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "0a4bde9a2fa89e8f1a1a086048e26155"
server_key_iv: "d0485eb1ab6eea4407bec8b5c5c93427"
server_key_version: "encryptionkey@0"
I20260810 06:46:54.117334 1666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 06:46:54.120754 1690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:54.122073 1666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:46:54.122269 1666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "6d6866c2413b4331a25a7d5e368dfb33"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "0a4bde9a2fa89e8f1a1a086048e26155"
server_key_iv: "d0485eb1ab6eea4407bec8b5c5c93427"
server_key_version: "encryptionkey@0"
I20260810 06:46:54.122453 1666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:54.144292 1666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:54.148375 1666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:54.148655 1666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:54.166417 1666 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:44249
I20260810 06:46:54.166455 1782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:44249 every 8 connection(s)
I20260810 06:46:54.167760 1666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:46:54.170892 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 1666
I20260810 06:46:54.171139 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:46:54.171895 1784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:54.171980 11208 external_mini_cluster.cc:1428] Setting key 2061f4b00582b4a53030224a62c84b7f
I20260810 06:46:54.184806 1784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Bootstrap starting.
Aug 10 06:46:54 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:54.188053 1784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:54.189218 1784 log.cc:824] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:54.192108 1784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: No bootstrap required, opened a new log
I20260810 06:46:54.194595 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:54.174210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54608 (local address 127.10.242.62:44249)
0810 06:46:54.175579 (+ 1369us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:54.175590 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:54.175633 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:46:54.177095 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:54.177116 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:54.177497 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:54.177845 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:54.179341 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:54.180606 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:54.181913 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:54.182654 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:54.182824 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:54.187295 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:54.187302 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:54.187315 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:54.187355 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:54.190650 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:54.191329 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:54.191335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:54.191341 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:54.191515 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:54.191906 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:54.191910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:54.191912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:54.192429 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:54.192758 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:54.193164 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 06:46:54.193551 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1160,"thread_start_us":933,"threads_started":1}
I20260810 06:46:54.199761 1784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:46:54.200090 1784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:54.200204 1784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d6866c2413b4331a25a7d5e368dfb33, State: Initialized, Role: FOLLOWER
I20260810 06:46:54.200721 1784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:46:54.200932 1784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:46:54.201061 1784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:46:54.201215 1784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:54.203616 1784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:46:54.204118 1784 leader_election.cc:302] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d6866c2413b4331a25a7d5e368dfb33; no voters:
I20260810 06:46:54.204681 1784 leader_election.cc:288] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:46:54.204806 1790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:54.205441 1790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 1 LEADER]: Becoming Leader. State: Replica: 6d6866c2413b4331a25a7d5e368dfb33, State: Running, Role: LEADER
I20260810 06:46:54.206007 1790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:46:54.206735 1784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:54.208559 1795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d6866c2413b4331a25a7d5e368dfb33. Latest consensus state: current_term: 1 leader_uuid: "6d6866c2413b4331a25a7d5e368dfb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } } }
I20260810 06:46:54.208833 1795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:54.209579 1794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d6866c2413b4331a25a7d5e368dfb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } } }
I20260810 06:46:54.209983 1794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:54.210686 1802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:46:54.215592 1802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:46:54.223727 1802 catalog_manager.cc:1347] Generated new cluster ID: 8c6a912fbe7b40bb9a6219a4d5a1e1a3
I20260810 06:46:54.223829 1802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:46:54.250761 1802 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:46:54.252661 1802 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:46:54.262181 1802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Generated new TSK 0
I20260810 06:46:54.263293 1802 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:46:54.327919 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:44249
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:54.660682 1831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:54.660948 1831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:54.661043 1831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:54.668080 1831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:54.668181 1831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:54.668289 1831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:46:54.677551 1831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:54.678767 1831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:54.680016 1831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:54.694550 1840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:54.694641 1841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:54.695276 1831 server_base.cc:1034] running on GCE node
W20260810 06:46:54.695493 1844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:54.696216 1831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:54.697304 1831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:54.698585 1831 hybrid_clock.cc:648] HybridClock initialized: now 1786344414698496 us; error 68 us; skew 500 ppm
I20260810 06:46:54.698930 1831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:54 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:54.701929 1831 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:46:54.703318 1831 webserver.cc:460] Webserver started at http://127.10.242.1:43897/ using document root <none> and password file <none>
I20260810 06:46:54.703960 1831 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:54.704078 1831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:54.704365 1831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:54.707309 1831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "cb594530a1404fda9e6440c0fa297e99"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "f21186b85906e89f41e33e495ef7453e"
server_key_iv: "85434ff2335cea78633b3954b1546a9e"
server_key_version: "encryptionkey@0"
I20260810 06:46:54.708060 1831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "cb594530a1404fda9e6440c0fa297e99"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "f21186b85906e89f41e33e495ef7453e"
server_key_iv: "85434ff2335cea78633b3954b1546a9e"
server_key_version: "encryptionkey@0"
I20260810 06:46:54.712961 1831 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 06:46:54.716168 1855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:54.717444 1831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:46:54.717587 1831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "cb594530a1404fda9e6440c0fa297e99"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "f21186b85906e89f41e33e495ef7453e"
server_key_iv: "85434ff2335cea78633b3954b1546a9e"
server_key_version: "encryptionkey@0"
I20260810 06:46:54.717780 1831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:54.731674 1831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:54.735527 1831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:54.735790 1831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:54.736683 1831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:54.737919 1831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:54.737998 1831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:54.738126 1831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:54.738206 1831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 06:46:54.768265 1831 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:46059
I20260810 06:46:54.768309 2025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:46059 every 8 connection(s)
I20260810 06:46:54.769521 1831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:46:54.773415 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 1831
I20260810 06:46:54.773645 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:46:54.774341 11208 external_mini_cluster.cc:1428] Setting key d83bac92732cc2b56bc9146374dd6f14
Aug 10 06:46:54 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:54.789315 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:54.771968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:33665 (local address 127.10.242.62:44249)
0810 06:46:54.772397 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:54.772402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:54.773309 (+ 907us) server_negotiation.cc:407] Connection header received
0810 06:46:54.775342 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:54.775348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:54.775429 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:54.775538 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:54.777035 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:54.777910 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:54.779990 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:54.780375 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:54.780478 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:54.784140 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:54.784145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:54.784148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:54.784180 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:54.786606 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:54.787267 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:54.787272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:54.787274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:54.787390 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:54.787811 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:54.787815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:54.787818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:54.788142 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:54.788282 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:54.788892 (+ 610us) server_negotiation.cc:299] Negotiation successful
0810 06:46:54.789052 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":314,"thread_start_us":211,"threads_started":1}
I20260810 06:46:54.790117 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:54.772468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:33665)
0810 06:46:54.773164 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:54.773201 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:54.775023 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:54.775730 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:54.775738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:54.776326 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:54.776874 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:54.776891 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:54.778066 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:54.778072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:54.779841 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:54.779858 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:54.780596 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:54.780603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:54.780873 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:54.781514 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:54.781541 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:54.783935 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:54.786729 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:54.786737 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:54.786752 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:54.787150 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:54.787510 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:54.787513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:54.787516 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:54.787703 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:54.788249 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:54.788254 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:54.788540 (+ 286us) client_negotiation.cc:769] Sending connection context
0810 06:46:54.788756 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 06:46:54.789106 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":264,"threads_started":1}
I20260810 06:46:54.791622 2026 heartbeater.cc:347] Connected to a master server at 127.10.242.62:44249
I20260810 06:46:54.791908 2026 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:54.792603 2026 heartbeater.cc:511] Master 127.10.242.62:44249 requested a full tablet report, sending...
I20260810 06:46:54.794857 1730 ts_manager.cc:194] Registered new tserver with Master: cb594530a1404fda9e6440c0fa297e99 (127.10.242.1:46059)
I20260810 06:46:54.796765 1730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:33665
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:46:54.836666 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:44249
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:55.161363 2036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:55.161638 2036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:55.161755 2036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:55.168890 2036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:55.169018 2036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:55.169188 2036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:46:55.178575 2036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:55.179744 2036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:55.181020 2036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:55.193552 2042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:55.193600 2043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:55.193543 2046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:55.194473 2036 server_base.cc:1034] running on GCE node
I20260810 06:46:55.195057 2036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:55.195914 2036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:55.197135 2036 hybrid_clock.cc:648] HybridClock initialized: now 1786344415197075 us; error 36 us; skew 500 ppm
I20260810 06:46:55.197430 2036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:55 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:55.200452 2036 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:46:55.201867 2036 webserver.cc:460] Webserver started at http://127.10.242.2:41597/ using document root <none> and password file <none>
I20260810 06:46:55.202610 2036 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:55.202714 2036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:55.202970 2036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:55.205817 2036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "263ee39503434734843ace6674894605"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "d4af9537a8b2aec0a823f8c96e70e07a"
server_key_iv: "5a7a4f409ba91d48d49d471ff3ee40d8"
server_key_version: "encryptionkey@0"
I20260810 06:46:55.206609 2036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "263ee39503434734843ace6674894605"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "d4af9537a8b2aec0a823f8c96e70e07a"
server_key_iv: "5a7a4f409ba91d48d49d471ff3ee40d8"
server_key_version: "encryptionkey@0"
I20260810 06:46:55.211750 2036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:46:55.215099 2062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:55.216411 2036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:46:55.216542 2036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "263ee39503434734843ace6674894605"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "d4af9537a8b2aec0a823f8c96e70e07a"
server_key_iv: "5a7a4f409ba91d48d49d471ff3ee40d8"
server_key_version: "encryptionkey@0"
I20260810 06:46:55.216715 2036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:55.238087 2036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:55.241962 2036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:55.242270 2036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:55.243206 2036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:55.244552 2036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:55.244634 2036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:55.244752 2036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:55.244805 2036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:55.274457 2036 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:37233
I20260810 06:46:55.274519 2271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:37233 every 8 connection(s)
I20260810 06:46:55.275661 2036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:46:55.280763 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 2036
I20260810 06:46:55.280929 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:46:55.281358 11208 external_mini_cluster.cc:1428] Setting key fe85bf1d829884ea8209d2e3445aca50
Aug 10 06:46:55 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:55.295773 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.278041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:32925 (local address 127.10.242.62:44249)
0810 06:46:55.278257 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:55.278262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:55.279330 (+ 1068us) server_negotiation.cc:407] Connection header received
0810 06:46:55.281148 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:55.281154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:55.281242 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:55.281332 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:55.283150 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.284094 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.285754 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.286369 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.286524 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.289810 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:55.289816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:55.289821 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:55.289859 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:55.292912 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:55.293596 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:55.293602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:55.293604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:55.293776 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:55.294138 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:55.294142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:55.294145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:55.294512 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:55.294650 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:55.295359 (+ 709us) server_negotiation.cc:299] Negotiation successful
0810 06:46:55.295530 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:46:55.297140 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.278480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:32925)
0810 06:46:55.279170 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:55.279215 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:55.280854 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:55.281571 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:55.281579 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:55.282240 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:55.282961 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.282993 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.284249 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.284253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.285493 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.285506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.286635 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.286643 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.286874 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.287495 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:55.287518 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:55.289565 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:55.293054 (+ 3489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:55.293061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:55.293072 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:55.293486 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:55.293869 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:55.293872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:55.293875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:55.294044 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:55.294660 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:55.294666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:55.294973 (+ 307us) client_negotiation.cc:769] Sending connection context
0810 06:46:55.295258 (+ 285us) client_negotiation.cc:240] Negotiation successful
0810 06:46:55.295772 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":258,"threads_started":1}
I20260810 06:46:55.298596 2275 heartbeater.cc:347] Connected to a master server at 127.10.242.62:44249
I20260810 06:46:55.298884 2275 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:55.299593 2275 heartbeater.cc:511] Master 127.10.242.62:44249 requested a full tablet report, sending...
I20260810 06:46:55.301097 1730 ts_manager.cc:194] Registered new tserver with Master: 263ee39503434734843ace6674894605 (127.10.242.2:37233)
I20260810 06:46:55.302201 1730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:32925
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:46:55.340972 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:44249
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:55.664948 2282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:55.665220 2282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:55.665313 2282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:55.672293 2282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:55.672396 2282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:55.672515 2282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:46:55.681811 2282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:55.682998 2282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:55.684285 2282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:55.696619 2293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:55.696997 2294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:55.697032 2299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:55.697671 2282 server_base.cc:1034] running on GCE node
I20260810 06:46:55.698294 2282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:55.699141 2282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:55.700343 2282 hybrid_clock.cc:648] HybridClock initialized: now 1786344415700299 us; error 28 us; skew 500 ppm
I20260810 06:46:55.700639 2282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:55 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:55.703617 2282 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:46:55.704998 2282 webserver.cc:460] Webserver started at http://127.10.242.3:46329/ using document root <none> and password file <none>
I20260810 06:46:55.705636 2282 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:55.705754 2282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:55.706037 2282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:46:55.708801 2282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "5111de7999f94e7aaac7a157af082a04"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "e03197be1cbb7bdf3c7bdf34c5c5d98c"
server_key_iv: "cc8e957981c502cf8c2ec99958861931"
server_key_version: "encryptionkey@0"
I20260810 06:46:55.709522 2282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "5111de7999f94e7aaac7a157af082a04"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "e03197be1cbb7bdf3c7bdf34c5c5d98c"
server_key_iv: "cc8e957981c502cf8c2ec99958861931"
server_key_version: "encryptionkey@0"
I20260810 06:46:55.714315 2282 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260810 06:46:55.717440 2308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:55.718691 2282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:46:55.718840 2282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "5111de7999f94e7aaac7a157af082a04"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "e03197be1cbb7bdf3c7bdf34c5c5d98c"
server_key_iv: "cc8e957981c502cf8c2ec99958861931"
server_key_version: "encryptionkey@0"
I20260810 06:46:55.719012 2282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:55.749751 2282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:55.753839 2282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:55.754110 2282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:55.755100 2282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:55.756415 2282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:55.756491 2282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:55.756603 2282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:55.756670 2282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:55.787161 2282 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:36171
I20260810 06:46:55.787230 2516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:36171 every 8 connection(s)
I20260810 06:46:55.788410 2282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:46:55.795276 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 2282
I20260810 06:46:55.795450 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:46:55.795897 11208 external_mini_cluster.cc:1428] Setting key ca1bbd94369151f51651f51eefeff3a6
I20260810 06:46:55.800143 2026 heartbeater.cc:503] Master 127.10.242.62:44249 was elected leader, sending a full tablet report...
Aug 10 06:46:55 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344415, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:55.808414 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.790977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:56901 (local address 127.10.242.62:44249)
0810 06:46:55.791169 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:55.791174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:55.792243 (+ 1069us) server_negotiation.cc:407] Connection header received
0810 06:46:55.794052 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:55.794058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:55.794201 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:55.794297 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:55.795787 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.796708 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.798433 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.798859 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.798969 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.802789 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:55.802795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:55.802799 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:55.802840 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:55.805660 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:55.806368 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:55.806371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:55.806374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:55.806498 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:55.806922 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:55.806925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:55.806927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:55.807208 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:55.807317 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:55.807989 (+ 672us) server_negotiation.cc:299] Negotiation successful
0810 06:46:55.808174 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:46:55.809448 2520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.791424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:56901)
0810 06:46:55.792087 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:55.792133 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:55.793781 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:55.794545 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:55.794553 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:55.795030 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:55.795616 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.795637 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.796869 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.796872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.798288 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.798302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.799083 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.799091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.799343 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.799914 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:55.799936 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:55.802520 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:55.805783 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:55.805790 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:55.805802 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:55.806284 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:55.806586 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:55.806589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:55.806591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:55.806830 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:55.807312 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:55.807318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:55.807638 (+ 320us) client_negotiation.cc:769] Sending connection context
0810 06:46:55.807869 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 06:46:55.808330 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":259,"threads_started":1}
I20260810 06:46:55.810807 2518 heartbeater.cc:347] Connected to a master server at 127.10.242.62:44249
I20260810 06:46:55.811133 2518 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:55.811846 2518 heartbeater.cc:511] Master 127.10.242.62:44249 requested a full tablet report, sending...
I20260810 06:46:55.813145 1730 ts_manager.cc:194] Registered new tserver with Master: 5111de7999f94e7aaac7a157af082a04 (127.10.242.3:36171)
I20260810 06:46:55.813920 1730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:56901
I20260810 06:46:55.823179 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:46:55.839764 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.827563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.10.242.62:44249)
0810 06:46:55.827742 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:55.827748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:55.828130 (+ 382us) server_negotiation.cc:407] Connection header received
0810 06:46:55.828306 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:55.828311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:55.828372 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:55.828450 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:55.829831 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.830611 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.831965 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.832393 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.832500 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.834853 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:55.834858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:55.834861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:55.834892 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:55.837083 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:55.837750 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:55.837756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:55.837759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:55.837906 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:55.838326 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:55.838331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:55.838333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:55.838694 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:55.838929 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:55.839262 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 06:46:55.839461 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:46:55.843474 1730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:46:55.850103 1730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:46:55.884400 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.861575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.10.242.1:46059)
0810 06:46:55.875276 (+ 13701us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:55.875286 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:55.875318 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:46:55.875429 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:55.875438 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:55.875724 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:55.875923 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:55.877256 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.878907 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.881863 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.883274 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.883405 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.883639 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:55.883926 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:46:55.884084 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13501,"thread_start_us":13309,"threads_started":1}
I20260810 06:46:55.884459 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.861887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.10.242.3:36171)
0810 06:46:55.877516 (+ 15629us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:55.877524 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:55.877554 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:46:55.877630 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:55.877635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:55.877775 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:55.877934 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:55.878492 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.879902 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.881754 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.883024 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.883144 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.883387 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:55.883950 (+ 563us) server_negotiation.cc:299] Negotiation successful
0810 06:46:55.884109 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15463,"thread_start_us":15303,"threads_started":1}
I20260810 06:46:55.884513 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.861366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:57908)
0810 06:46:55.874280 (+ 12914us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:55.874334 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:55.874525 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:55.876462 (+ 1937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:55.876471 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:55.876510 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:55.877033 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.877048 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.879070 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.879075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.881680 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.881695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.883481 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.883488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.883688 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.883759 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 06:46:55.883865 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 06:46:55.884060 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12601,"mutex_wait_us":121,"thread_start_us":13876,"threads_started":1}
I20260810 06:46:55.885104 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.863135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33972 (local address 127.10.242.2:37233)
0810 06:46:55.876098 (+ 12963us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:55.876105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:55.876798 (+ 693us) server_negotiation.cc:407] Connection header received
0810 06:46:55.876901 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:55.876908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:55.877138 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:55.877325 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:55.878033 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.879579 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.882108 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.883503 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.883630 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.883823 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:55.884606 (+ 783us) server_negotiation.cc:299] Negotiation successful
0810 06:46:55.884783 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12804,"thread_start_us":12802,"threads_started":1}
I20260810 06:46:55.885206 2538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.861431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:36171 (local address 127.0.0.1:39068)
0810 06:46:55.875488 (+ 14057us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:55.875506 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:55.875620 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:55.877943 (+ 2323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:55.877947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:55.877969 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:55.878333 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.878344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.880040 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.880044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.881598 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.881610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.883541 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.883548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.883735 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.883766 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:46:55.883870 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 06:46:55.884878 (+ 1008us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13941,"mutex_wait_us":29,"thread_start_us":14725,"threads_started":1}
I20260810 06:46:55.885501 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:55.862626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:37233 (local address 127.0.0.1:33972)
0810 06:46:55.876520 (+ 13894us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:55.876538 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:55.876659 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:55.877473 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:55.877477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:55.877503 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:55.877826 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.877838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.879738 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.879743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.881958 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:55.881971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:55.883962 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:55.883968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:55.884124 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:55.884151 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 06:46:55.884240 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:46:55.884361 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13765,"mutex_wait_us":50,"thread_start_us":14186,"threads_started":1}
I20260810 06:46:55.888729 1921 tablet_service.cc:1467] Processing CreateTablet for tablet 2a7a5fae1d05449bb1f0423c048cdfd5 (DEFAULT_TABLE table=test-table [id=e94f5d0ed88c46329a7126b3808c5926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:55.888842 2394 tablet_service.cc:1467] Processing CreateTablet for tablet 2a7a5fae1d05449bb1f0423c048cdfd5 (DEFAULT_TABLE table=test-table [id=e94f5d0ed88c46329a7126b3808c5926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:55.888842 2147 tablet_service.cc:1467] Processing CreateTablet for tablet 2a7a5fae1d05449bb1f0423c048cdfd5 (DEFAULT_TABLE table=test-table [id=e94f5d0ed88c46329a7126b3808c5926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:55.889959 1921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a7a5fae1d05449bb1f0423c048cdfd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:55.890141 2147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a7a5fae1d05449bb1f0423c048cdfd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:55.890172 2394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a7a5fae1d05449bb1f0423c048cdfd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:55.901134 2552 tablet_bootstrap.cc:492] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: Bootstrap starting.
I20260810 06:46:55.903935 2555 tablet_bootstrap.cc:492] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: Bootstrap starting.
I20260810 06:46:55.905344 2552 tablet_bootstrap.cc:654] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:55.905799 2554 tablet_bootstrap.cc:492] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: Bootstrap starting.
I20260810 06:46:55.906896 2552 log.cc:824] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:55.909297 2554 tablet_bootstrap.cc:654] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:55.909617 2552 tablet_bootstrap.cc:492] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: No bootstrap required, opened a new log
I20260810 06:46:55.909864 2552 ts_tablet_manager.cc:1397] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:46:55.909863 2555 tablet_bootstrap.cc:654] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:55.910535 2554 log.cc:824] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:55.911047 2555 log.cc:824] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:55.913818 2555 tablet_bootstrap.cc:492] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: No bootstrap required, opened a new log
I20260810 06:46:55.914077 2555 ts_tablet_manager.cc:1397] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260810 06:46:55.914098 2554 tablet_bootstrap.cc:492] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: No bootstrap required, opened a new log
I20260810 06:46:55.914386 2554 ts_tablet_manager.cc:1397] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 06:46:55.919548 2552 raft_consensus.cc:348] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:55.919981 2552 raft_consensus.cc:374] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:55.920081 2552 raft_consensus.cc:729] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb594530a1404fda9e6440c0fa297e99, State: Initialized, Role: FOLLOWER
I20260810 06:46:55.920704 2552 consensus_queue.cc:260] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:55.920826 2555 raft_consensus.cc:348] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:55.921197 2555 raft_consensus.cc:374] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:55.921340 2555 raft_consensus.cc:729] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263ee39503434734843ace6674894605, State: Initialized, Role: FOLLOWER
I20260810 06:46:55.922041 2555 consensus_queue.cc:260] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:55.922560 2552 ts_tablet_manager.cc:1428] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260810 06:46:55.924400 2554 raft_consensus.cc:348] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:55.924729 2554 raft_consensus.cc:374] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:55.924829 2554 raft_consensus.cc:729] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5111de7999f94e7aaac7a157af082a04, State: Initialized, Role: FOLLOWER
I20260810 06:46:55.925338 2554 consensus_queue.cc:260] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:55.926419 2518 heartbeater.cc:503] Master 127.10.242.62:44249 was elected leader, sending a full tablet report...
I20260810 06:46:55.928112 2554 ts_tablet_manager.cc:1428] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260810 06:46:55.929574 2275 heartbeater.cc:503] Master 127.10.242.62:44249 was elected leader, sending a full tablet report...
I20260810 06:46:55.930469 2555 ts_tablet_manager.cc:1428] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
W20260810 06:46:56.021759 2027 tablet.cc:2369] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:46:56.027503 2277 tablet.cc:2369] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:46:56.040015 2519 tablet.cc:2369] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:46:56.224737 2561 raft_consensus.cc:482] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:46:56.225028 2561 raft_consensus.cc:504] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:56.226940 2561 leader_election.cc:288] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5111de7999f94e7aaac7a157af082a04 (127.10.242.3:36171), 263ee39503434734843ace6674894605 (127.10.242.2:37233)
I20260810 06:46:56.233660 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:56.227356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:43719 (local address 127.10.242.2:37233)
0810 06:46:56.227573 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:56.227580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:56.228353 (+ 773us) server_negotiation.cc:407] Connection header received
0810 06:46:56.228587 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:56.228592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:56.228661 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:56.228749 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:56.229340 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.230034 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.231996 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.232763 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.232898 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:56.232974 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:56.233287 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 06:46:56.233410 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:46:56.233745 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:56.227522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:37403 (local address 127.10.242.3:36171)
0810 06:46:56.227746 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:56.227754 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:56.228670 (+ 916us) server_negotiation.cc:407] Connection header received
0810 06:46:56.228815 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:56.228819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:56.228877 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:56.228948 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:56.229501 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.230410 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.232131 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.232876 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.233033 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:56.233105 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:56.233367 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 06:46:56.233492 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 06:46:56.233804 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:56.227366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:36171 (local address 127.10.242.1:37403)
0810 06:46:56.228575 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:56.228588 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:56.228694 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:56.228988 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:56.228992 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:56.229017 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:56.229361 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:56.229370 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.230543 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.230547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:56.231984 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:56.231997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.233088 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.233095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:56.233228 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:56.233251 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:46:56.233326 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 06:46:56.233443 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"thread_start_us":538,"threads_started":1}
I20260810 06:46:56.233947 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:56.227244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:37233 (local address 127.10.242.1:43719)
0810 06:46:56.228231 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:56.228256 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:56.228432 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:56.228789 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:56.228793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:56.228821 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:56.229186 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:56.229200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.230202 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.230207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:56.231838 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:56.231851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.232971 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.232978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:56.233134 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:56.233157 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:46:56.233237 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:46:56.233357 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":3,"thread_start_us":1034,"threads_started":1}
I20260810 06:46:56.234892 2427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7a5fae1d05449bb1f0423c048cdfd5" candidate_uuid: "cb594530a1404fda9e6440c0fa297e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5111de7999f94e7aaac7a157af082a04" is_pre_election: true
I20260810 06:46:56.234892 2182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7a5fae1d05449bb1f0423c048cdfd5" candidate_uuid: "cb594530a1404fda9e6440c0fa297e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ee39503434734843ace6674894605" is_pre_election: true
I20260810 06:46:56.235455 2427 raft_consensus.cc:2454] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb594530a1404fda9e6440c0fa297e99 in term 0.
I20260810 06:46:56.235450 2182 raft_consensus.cc:2454] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb594530a1404fda9e6440c0fa297e99 in term 0.
I20260810 06:46:56.236060 1857 leader_election.cc:302] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5111de7999f94e7aaac7a157af082a04, cb594530a1404fda9e6440c0fa297e99; no voters:
I20260810 06:46:56.236399 2561 raft_consensus.cc:2781] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:46:56.236536 2561 raft_consensus.cc:482] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:46:56.236642 2561 raft_consensus.cc:3037] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:56.238854 2561 raft_consensus.cc:504] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:56.239871 2561 leader_election.cc:288] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [CANDIDATE]: Term 1 election: Requested vote from peers 5111de7999f94e7aaac7a157af082a04 (127.10.242.3:36171), 263ee39503434734843ace6674894605 (127.10.242.2:37233)
I20260810 06:46:56.239848 2427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7a5fae1d05449bb1f0423c048cdfd5" candidate_uuid: "cb594530a1404fda9e6440c0fa297e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5111de7999f94e7aaac7a157af082a04"
I20260810 06:46:56.239979 2182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a7a5fae1d05449bb1f0423c048cdfd5" candidate_uuid: "cb594530a1404fda9e6440c0fa297e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ee39503434734843ace6674894605"
I20260810 06:46:56.240056 2427 raft_consensus.cc:3037] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:56.240186 2182 raft_consensus.cc:3037] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:56.243274 2427 raft_consensus.cc:2454] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb594530a1404fda9e6440c0fa297e99 in term 1.
I20260810 06:46:56.243274 2182 raft_consensus.cc:2454] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb594530a1404fda9e6440c0fa297e99 in term 1.
I20260810 06:46:56.243673 1856 leader_election.cc:302] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 263ee39503434734843ace6674894605, cb594530a1404fda9e6440c0fa297e99; no voters:
I20260810 06:46:56.243924 2561 raft_consensus.cc:2781] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:56.244379 2561 raft_consensus.cc:686] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 1 LEADER]: Becoming Leader. State: Replica: cb594530a1404fda9e6440c0fa297e99, State: Running, Role: LEADER
I20260810 06:46:56.244985 2561 consensus_queue.cc:237] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:56.249786 1728 catalog_manager.cc:5495] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb594530a1404fda9e6440c0fa297e99 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "cb594530a1404fda9e6440c0fa297e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } health_report { overall_health: UNKNOWN } } }
I20260810 06:46:56.342540 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:56.337237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57910 (local address 127.10.242.1:46059)
0810 06:46:56.337458 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:56.337463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:56.337483 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:46:56.337593 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:56.337599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:56.337732 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:56.337877 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:46:56.338253 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.338968 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.340391 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.340812 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.340921 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:56.341448 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:46:56.341952 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:46:56.342088 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:56.342208 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 06:46:56.342291 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:46:56.350417 2427 raft_consensus.cc:1261] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [term 1 FOLLOWER]: Refusing update from remote peer cb594530a1404fda9e6440c0fa297e99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:56.350414 2182 raft_consensus.cc:1261] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [term 1 FOLLOWER]: Refusing update from remote peer cb594530a1404fda9e6440c0fa297e99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:56.351326 2567 consensus_queue.cc:1035] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:56.351649 2561 consensus_queue.cc:1035] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:56.361580 2570 mvcc.cc:205] Tried to move back new op lower bound from 7316866729361666048 to 7316866728943595520. Current Snapshot: MvccSnapshot[applied={T|T < 7316866729361666048}]
I20260810 06:46:56.362929 2571 mvcc.cc:205] Tried to move back new op lower bound from 7316866729361666048 to 7316866728943595520. Current Snapshot: MvccSnapshot[applied={T|T < 7316866729361666048}]
I20260810 06:46:56.373869 1730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:56.374135 1730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:56.377524 1730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Sending DeleteTablet for 3 replicas of tablet 2a7a5fae1d05449bb1f0423c048cdfd5
I20260810 06:46:56.378623 2147 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a7a5fae1d05449bb1f0423c048cdfd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:56 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:33972
I20260810 06:46:56.378602 1921 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a7a5fae1d05449bb1f0423c048cdfd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:56 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:57908
I20260810 06:46:56.378643 2394 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a7a5fae1d05449bb1f0423c048cdfd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:56 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:39068
I20260810 06:46:56.379931 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 1831
I20260810 06:46:56.394637 2591 tablet_replica.cc:330] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: stopping tablet replica
I20260810 06:46:56.395219 2591 raft_consensus.cc:2229] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:46:56.395825 2591 raft_consensus.cc:2258] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:56.398218 2591 ts_tablet_manager.cc:1905] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:56.405067 2591 ts_tablet_manager.cc:1918] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:56.405637 2591 log.cc:1196] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/2a7a5fae1d05449bb1f0423c048cdfd5
I20260810 06:46:56.406304 2591 ts_tablet_manager.cc:1939] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 5111de7999f94e7aaac7a157af082a04: Deleting consensus metadata
I20260810 06:46:56.408254 1695 catalog_manager.cc:4841] TS 5111de7999f94e7aaac7a157af082a04 (127.10.242.3:36171): tablet 2a7a5fae1d05449bb1f0423c048cdfd5 (table test-table [id=e94f5d0ed88c46329a7126b3808c5926]) successfully deleted
I20260810 06:46:56.411144 2590 tablet_replica.cc:330] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: stopping tablet replica
I20260810 06:46:56.411391 2587 tablet_replica.cc:330] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: stopping tablet replica
I20260810 06:46:56.411772 2590 raft_consensus.cc:2229] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:46:56.411859 2587 raft_consensus.cc:2229] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:46:56.412372 2590 raft_consensus.cc:2258] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:56.412575 2587 raft_consensus.cc:2258] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:56.414299 2590 ts_tablet_manager.cc:1905] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:56.415032 2587 ts_tablet_manager.cc:1905] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:46:56 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:46:56.421036 2590 ts_tablet_manager.cc:1918] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:56.421234 2590 log.cc:1196] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/2a7a5fae1d05449bb1f0423c048cdfd5
I20260810 06:46:56.421838 2590 ts_tablet_manager.cc:1939] T 2a7a5fae1d05449bb1f0423c048cdfd5 P 263ee39503434734843ace6674894605: Deleting consensus metadata
I20260810 06:46:56.423218 2587 ts_tablet_manager.cc:1918] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:56.423801 1692 catalog_manager.cc:4841] TS 263ee39503434734843ace6674894605 (127.10.242.2:37233): tablet 2a7a5fae1d05449bb1f0423c048cdfd5 (table test-table [id=e94f5d0ed88c46329a7126b3808c5926]) successfully deleted
I20260810 06:46:56.423506 2587 log.cc:1196] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/2a7a5fae1d05449bb1f0423c048cdfd5
I20260810 06:46:56.424575 2587 ts_tablet_manager.cc:1939] T 2a7a5fae1d05449bb1f0423c048cdfd5 P cb594530a1404fda9e6440c0fa297e99: Deleting consensus metadata
I20260810 06:46:56.425186 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:56.381214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.10.242.1:46059)
0810 06:46:56.381395 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:56.381402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:56.403754 (+ 22352us) server_negotiation.cc:407] Connection header received
0810 06:46:56.404265 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:56.404272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:56.404365 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:56.404438 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:56.406647 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.407986 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.410454 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.411211 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.411307 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:56.417156 (+ 5849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:56.417170 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:56.417192 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:56.417239 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:56.421467 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:56.422366 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:56.422376 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:56.422379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:56.422613 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:56.423135 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:56.423141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:56.423144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:56.423618 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:56.423898 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:56.424508 (+ 610us) server_negotiation.cc:299] Negotiation successful
0810 06:46:56.424764 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:46:56.426057 2020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:56.426362 1696 catalog_manager.cc:4841] TS cb594530a1404fda9e6440c0fa297e99 (127.10.242.1:46059): tablet 2a7a5fae1d05449bb1f0423c048cdfd5 (table test-table [id=e94f5d0ed88c46329a7126b3808c5926]) successfully deleted
I20260810 06:46:56.523614 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 1831
I20260810 06:46:56.550674 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 2036
Aug 10 06:46:56 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:46:56.565550 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:56.551460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.10.242.2:37233)
0810 06:46:56.551644 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:56.551650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:56.551670 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:46:56.551828 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:56.551832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:56.551908 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:56.551994 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:56.553310 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.554028 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.555331 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.555746 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.555858 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:56.559993 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:56.560010 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:56.560028 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:56.560062 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:56.562775 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:56.563437 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:56.563445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:56.563448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:56.563589 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:56.563949 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:56.563953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:56.563954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:56.564360 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:56.564644 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:56.564985 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 06:46:56.565207 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:46:56.566234 2265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:56.653313 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 2036
I20260810 06:46:56.678318 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 2282
Aug 10 06:46:56 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:46:56.692917 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:56.679076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.10.242.3:36171)
0810 06:46:56.679269 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:56.679274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:56.679295 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:46:56.679437 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:56.679441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:56.679507 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:56.679587 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:56.680799 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.681514 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.682842 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:56.683239 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:56.683328 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:56.687488 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:56.687505 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:56.687523 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:56.687556 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:56.690111 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:56.690765 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:56.690775 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:56.690778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:56.690949 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:56.691261 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:56.691265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:56.691268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:56.691689 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:56.691964 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:56.692158 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 06:46:56.692379 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:46:56.693531 2512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:56.780201 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 2282
I20260810 06:46:56.804348 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 1666
I20260810 06:46:56.805649 1776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:56.916643 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 1666
I20260810 06:46:56.947445 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:44249
--webserver_interface=127.10.242.62
--webserver_port=40279
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:44249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:57.276050 2602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:57.276355 2602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:57.276502 2602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:57.283469 2602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:57.283576 2602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:57.283655 2602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:57.283910 2602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:57.283985 2602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:57.293896 2602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:44249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:44249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=40279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:57.295152 2602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:57.296407 2602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:57.320248 2615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:57.320281 2613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:57.320307 2619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:57.320988 2602 server_base.cc:1034] running on GCE node
I20260810 06:46:57.321774 2602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:57.323298 2602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:57.324496 2602 hybrid_clock.cc:648] HybridClock initialized: now 1786344417324455 us; error 30 us; skew 500 ppm
I20260810 06:46:57.324862 2602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:57 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344417, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:57.328333 2602 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:46:57.329905 2602 webserver.cc:460] Webserver started at http://127.10.242.62:40279/ using document root <none> and password file <none>
I20260810 06:46:57.330693 2602 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:57.330819 2602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:57.336838 2602 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 06:46:57.339674 2634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:57.340955 2602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:46:57.341104 2602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "6d6866c2413b4331a25a7d5e368dfb33"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "0a4bde9a2fa89e8f1a1a086048e26155"
server_key_iv: "d0485eb1ab6eea4407bec8b5c5c93427"
server_key_version: "encryptionkey@0"
I20260810 06:46:57.342126 2602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:57.372699 2602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:57.376686 2602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:57.376915 2602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:57.394629 2602 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:44249
I20260810 06:46:57.394659 2721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:44249 every 8 connection(s)
I20260810 06:46:57.396042 2602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:46:57.401232 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 2602
I20260810 06:46:57.402294 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:46059
--local_ip_for_outbound_sockets=127.10.242.1
--tserver_master_addrs=127.10.242.62:44249
--webserver_port=43897
--webserver_interface=127.10.242.1
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:46:57.403502 2722 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:46:57.407863 2722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Bootstrap starting.
I20260810 06:46:57.428369 2722 log.cc:824] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:57.440604 2722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:46:57.441093 2722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Bootstrap complete.
I20260810 06:46:57.450301 2722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:46:57.451117 2722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d6866c2413b4331a25a7d5e368dfb33, State: Initialized, Role: FOLLOWER
I20260810 06:46:57.451704 2722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:46:57.451929 2722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:46:57.452040 2722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:46:57.452199 2722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:46:57.455902 2722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:46:57.456385 2722 leader_election.cc:302] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d6866c2413b4331a25a7d5e368dfb33; no voters:
I20260810 06:46:57.456962 2722 leader_election.cc:288] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:46:57.457093 2729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:46:57.458254 2729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 2 LEADER]: Becoming Leader. State: Replica: 6d6866c2413b4331a25a7d5e368dfb33, State: Running, Role: LEADER
I20260810 06:46:57.458846 2729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:46:57.459174 2722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:46:57.461273 2737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d6866c2413b4331a25a7d5e368dfb33. Latest consensus state: current_term: 2 leader_uuid: "6d6866c2413b4331a25a7d5e368dfb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } } }
I20260810 06:46:57.461673 2737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:57.462221 2732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6d6866c2413b4331a25a7d5e368dfb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } } }
I20260810 06:46:57.462509 2732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: This master's current role is: LEADER
I20260810 06:46:57.472513 2744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:46:57.480870 2744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:46:57.482393 2744 catalog_manager.cc:1259] Loaded cluster ID: 8c6a912fbe7b40bb9a6219a4d5a1e1a3
I20260810 06:46:57.482546 2744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:46:57.485895 2744 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 06:46:57.505488 2744 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 06:46:57.514770 2744 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:46:57.515997 2744 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Loaded TSK: 0
I20260810 06:46:57.517033 2744 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:46:57.762041 2725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:57.762349 2725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:57.762511 2725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:57.770021 2725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:57.770145 2725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:57.770324 2725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:46:57.780151 2725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:46059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=43897
--tserver_master_addrs=127.10.242.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:57.781385 2725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:57.782778 2725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:57.797344 2763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:57.797585 2762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:57.797926 2768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:57.798249 2725 server_base.cc:1034] running on GCE node
I20260810 06:46:57.798988 2725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:57.800164 2725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:57.801421 2725 hybrid_clock.cc:648] HybridClock initialized: now 1786344417801361 us; error 47 us; skew 500 ppm
I20260810 06:46:57.801769 2725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:57 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344417, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:57.805471 2725 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:46:57.807098 2725 webserver.cc:460] Webserver started at http://127.10.242.1:43897/ using document root <none> and password file <none>
I20260810 06:46:57.807966 2725 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:57.808127 2725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:57.814797 2725 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:46:57.817992 2784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:57.819531 2725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:46:57.819689 2725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "cb594530a1404fda9e6440c0fa297e99"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "f21186b85906e89f41e33e495ef7453e"
server_key_iv: "85434ff2335cea78633b3954b1546a9e"
server_key_version: "encryptionkey@0"
I20260810 06:46:57.820994 2725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:57.851230 2725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:57.856251 2725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:57.856555 2725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:57.857530 2725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:57.859017 2725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:57.859123 2725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:57.859208 2725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:57.859262 2725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:57.888104 2725 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:46059
I20260810 06:46:57.888168 2972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:46059 every 8 connection(s)
I20260810 06:46:57.889482 2725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:46:57.890937 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 2725
I20260810 06:46:57.891686 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:37233
--local_ip_for_outbound_sockets=127.10.242.2
--tserver_master_addrs=127.10.242.62:44249
--webserver_port=41597
--webserver_interface=127.10.242.2
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:46:57 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344417, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:57.913259 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:57.892521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:57041)
0810 06:46:57.893434 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:57.893489 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:57.895618 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:57.896912 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:57.896921 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:57.897610 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:57.898368 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:57.898394 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:57.899970 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:57.899975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:57.901675 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:57.901691 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:57.902705 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:57.902713 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:57.902967 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:57.903566 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:57.903589 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:57.905834 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:57.909083 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:57.909092 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:57.909107 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:57.909530 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:57.909955 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:57.909960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:57.909964 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:57.910145 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:57.910866 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:57.910874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:57.911196 (+ 322us) client_negotiation.cc:769] Sending connection context
0810 06:46:57.911425 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 06:46:57.911970 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":353,"threads_started":1}
I20260810 06:46:57.913260 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:57.892249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:57041 (local address 127.10.242.62:44249)
0810 06:46:57.894306 (+ 2057us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:57.894317 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:57.894359 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:46:57.896095 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:57.896107 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:57.896402 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:57.896694 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:57.898578 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:57.899794 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:57.901797 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:57.902466 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:57.902652 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:57.906081 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:57.906088 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:57.906102 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:57.906146 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:57.908942 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:57.909664 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:57.909671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:57.909677 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:57.909842 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:57.910276 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:57.910282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:57.910284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:57.910730 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:57.910915 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:57.911671 (+ 756us) server_negotiation.cc:299] Negotiation successful
0810 06:46:57.912039 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1857,"thread_start_us":1720,"threads_started":1}
I20260810 06:46:57.916389 2974 heartbeater.cc:347] Connected to a master server at 127.10.242.62:44249
I20260810 06:46:57.916669 2974 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:57.917371 2974 heartbeater.cc:511] Master 127.10.242.62:44249 requested a full tablet report, sending...
I20260810 06:46:57.919766 2666 ts_manager.cc:194] Registered new tserver with Master: cb594530a1404fda9e6440c0fa297e99 (127.10.242.1:46059)
I20260810 06:46:57.921922 2666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:57041
W20260810 06:46:58.222811 2980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:58.223083 2980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:58.223182 2980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:58.230274 2980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:58.230373 2980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:58.230492 2980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:46:58.239938 2980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:37233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=41597
--tserver_master_addrs=127.10.242.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:58.241176 2980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:58.242492 2980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:58.255434 2990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:58.255508 2996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:58.255475 2991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:58.256253 2980 server_base.cc:1034] running on GCE node
I20260810 06:46:58.256868 2980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:58.257746 2980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:58.258941 2980 hybrid_clock.cc:648] HybridClock initialized: now 1786344418258889 us; error 40 us; skew 500 ppm
I20260810 06:46:58.259245 2980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:58 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344418, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:58.262293 2980 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:46:58.263665 2980 webserver.cc:460] Webserver started at http://127.10.242.2:41597/ using document root <none> and password file <none>
I20260810 06:46:58.264300 2980 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:58.264412 2980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:58.269570 2980 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:46:58.272198 3011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:58.273471 2980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:46:58.273624 2980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "263ee39503434734843ace6674894605"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "d4af9537a8b2aec0a823f8c96e70e07a"
server_key_iv: "5a7a4f409ba91d48d49d471ff3ee40d8"
server_key_version: "encryptionkey@0"
I20260810 06:46:58.274629 2980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:58.321808 2980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:58.325784 2980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:58.326028 2980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:58.327128 2980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:58.328480 2980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:58.328564 2980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:58.328652 2980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:58.328709 2980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:58.358503 2980 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:37233
I20260810 06:46:58.358575 3226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:37233 every 8 connection(s)
I20260810 06:46:58.359946 2980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:46:58.370288 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 2980
I20260810 06:46:58.370874 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:36171
--local_ip_for_outbound_sockets=127.10.242.3
--tserver_master_addrs=127.10.242.62:44249
--webserver_port=46329
--webserver_interface=127.10.242.3
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:46:58 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344418, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:58.379570 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.362379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:50121 (local address 127.10.242.62:44249)
0810 06:46:58.362542 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:58.362547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:58.363686 (+ 1139us) server_negotiation.cc:407] Connection header received
0810 06:46:58.365474 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:58.365480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:58.365578 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:58.365667 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:58.367202 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.367891 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.369424 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.369876 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.369985 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.373528 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:58.373534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:58.373539 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:58.373578 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:58.376516 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:58.377210 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:58.377216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:58.377220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:58.377387 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:58.377818 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:58.377822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:58.377825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:58.378197 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:58.378332 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:58.379017 (+ 685us) server_negotiation.cc:299] Negotiation successful
0810 06:46:58.379251 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:46:58.380380 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.362806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:50121)
0810 06:46:58.363522 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:58.363560 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:58.365209 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:58.365922 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:58.365930 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:58.366458 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:58.367021 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.367039 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.368050 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.368054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.369273 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.369286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.370090 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.370098 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.370363 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.370996 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:58.371029 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:58.373322 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:58.376659 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:58.376666 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:58.376677 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:58.377081 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:58.377489 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:58.377492 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:58.377495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:58.377709 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:58.378305 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:58.378310 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:58.378625 (+ 315us) client_negotiation.cc:769] Sending connection context
0810 06:46:58.378885 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 06:46:58.379294 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":265,"threads_started":1}
I20260810 06:46:58.381781 3227 heartbeater.cc:347] Connected to a master server at 127.10.242.62:44249
I20260810 06:46:58.382125 3227 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:58.382874 3227 heartbeater.cc:511] Master 127.10.242.62:44249 requested a full tablet report, sending...
I20260810 06:46:58.384274 2666 ts_manager.cc:194] Registered new tserver with Master: 263ee39503434734843ace6674894605 (127.10.242.2:37233)
I20260810 06:46:58.385063 2666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:50121
W20260810 06:46:58.697453 3231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:58.697721 3231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:58.697860 3231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:58.705120 3231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:58.705232 3231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:58.705392 3231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:46:58.714555 3231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:36171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=46329
--tserver_master_addrs=127.10.242.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:58.715687 3231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:58.716981 3231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:46:58.729588 3240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:58.729703 3244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:46:58.729776 3239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:46:58.730458 3231 server_base.cc:1034] running on GCE node
I20260810 06:46:58.731047 3231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:46:58.731917 3231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:46:58.733098 3231 hybrid_clock.cc:648] HybridClock initialized: now 1786344418733045 us; error 44 us; skew 500 ppm
I20260810 06:46:58.733367 3231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:46:58 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344418, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:46:58.736320 3231 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:46:58.737565 3231 webserver.cc:460] Webserver started at http://127.10.242.3:46329/ using document root <none> and password file <none>
I20260810 06:46:58.738144 3231 fs_manager.cc:359] Metadata directory not provided
I20260810 06:46:58.738300 3231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:46:58.743166 3231 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:46:58.745700 3254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:58.746996 3231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:46:58.747128 3231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "5111de7999f94e7aaac7a157af082a04"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "e03197be1cbb7bdf3c7bdf34c5c5d98c"
server_key_iv: "cc8e957981c502cf8c2ec99958861931"
server_key_version: "encryptionkey@0"
I20260810 06:46:58.748037 3231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:46:58.766780 3231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:46:58.770347 3231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:46:58.770555 3231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:46:58.771400 3231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:46:58.772647 3231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:46:58.772732 3231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:58.772845 3231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:46:58.772909 3231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:46:58.801741 3231 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:36171
I20260810 06:46:58.801806 3457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:36171 every 8 connection(s)
I20260810 06:46:58.803146 3231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:46:58.808562 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 3231
Aug 10 06:46:58 dist-test-slave-zj8q krb5kdc[1647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344418, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:46:58.824762 2977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.805551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:56105 (local address 127.10.242.62:44249)
0810 06:46:58.805750 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:58.805754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:58.806810 (+ 1056us) server_negotiation.cc:407] Connection header received
0810 06:46:58.808770 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:58.808775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:58.808844 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:58.808930 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:58.810796 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.811850 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.814195 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.814830 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.814959 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.818478 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:58.818483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:58.818486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:58.818517 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:58.821324 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:58.822193 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:58.822198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:58.822201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:58.822370 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:58.822785 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:58.822789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:58.822792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:58.823126 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:58.823283 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:58.824062 (+ 779us) server_negotiation.cc:299] Negotiation successful
0810 06:46:58.824306 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:46:58.825027 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.809543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54626 (local address 127.10.242.62:44249)
0810 06:46:58.810027 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:58.810033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:58.810572 (+ 539us) server_negotiation.cc:407] Connection header received
0810 06:46:58.810745 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:58.810750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:58.810821 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:58.810942 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:58.812869 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.813995 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.815766 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.816197 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.816290 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.819093 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:58.819100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:58.819104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:58.819152 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:58.822113 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:58.822894 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:58.822897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:58.822899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:58.823016 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:58.823422 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:58.823426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:58.823429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:58.823778 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:58.824029 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:58.824398 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 06:46:58.824609 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":244,"threads_started":1}
I20260810 06:46:58.825870 3460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.805956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:56105)
0810 06:46:58.806644 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:58.806690 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:58.808420 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:58.809208 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:58.809216 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:58.809824 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:46:58.810620 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.810647 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.812064 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.812070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.814040 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.814057 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.815049 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.815054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.815256 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.815881 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:46:58.815909 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:46:58.818230 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:46:58.821458 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:58.821467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:58.821484 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:58.822033 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:58.822475 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:46:58.822478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:46:58.822480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:46:58.822660 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:46:58.823252 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:46:58.823258 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:46:58.823640 (+ 382us) client_negotiation.cc:769] Sending connection context
0810 06:46:58.823935 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 06:46:58.824415 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":280,"threads_started":1}
I20260810 06:46:58.827230 3458 heartbeater.cc:347] Connected to a master server at 127.10.242.62:44249
I20260810 06:46:58.827533 3458 heartbeater.cc:464] Registering TS with master...
I20260810 06:46:58.828315 3458 heartbeater.cc:511] Master 127.10.242.62:44249 requested a full tablet report, sending...
I20260810 06:46:58.829684 2665 ts_manager.cc:194] Registered new tserver with Master: 5111de7999f94e7aaac7a157af082a04 (127.10.242.3:36171)
I20260810 06:46:58.830556 2665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:56105
I20260810 06:46:58.839874 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:46:58.846624 3462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.840753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.10.242.62:44249)
0810 06:46:58.840932 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:58.840939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:58.841357 (+ 418us) server_negotiation.cc:407] Connection header received
0810 06:46:58.841514 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:58.841518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:58.841652 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:58.841783 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:46:58.842170 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.842885 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.844518 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.844952 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.845039 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.845498 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:46:58.846037 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:46:58.846142 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:58.846318 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 06:46:58.846385 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:46:58.847517 2665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:46:58.854847 2665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:46:58.892871 3475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.866188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.10.242.3:36171)
0810 06:46:58.882354 (+ 16166us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:58.882364 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:58.883669 (+ 1305us) server_negotiation.cc:407] Connection header received
0810 06:46:58.884116 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:58.884126 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:58.884397 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:58.884568 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:58.885676 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.887195 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.889989 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.891436 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.891536 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.891765 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:58.892411 (+ 646us) server_negotiation.cc:299] Negotiation successful
0810 06:46:58.892562 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15974,"thread_start_us":15606,"threads_started":1}
I20260810 06:46:58.893520 3474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.866010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:36171 (local address 127.0.0.1:39086)
0810 06:46:58.883485 (+ 17475us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:58.883531 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:58.883855 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:58.884968 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:58.884975 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:58.885008 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:58.885497 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.885509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.887365 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.887371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.889782 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.889801 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.892014 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.892022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.892191 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.892226 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:46:58.892334 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 06:46:58.892505 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17194,"thread_start_us":17041,"threads_started":1}
I20260810 06:46:58.894429 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.866869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57932 (local address 127.10.242.1:46059)
0810 06:46:58.882843 (+ 15974us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:58.882853 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:58.884708 (+ 1855us) server_negotiation.cc:407] Connection header received
0810 06:46:58.885589 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:58.885596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:58.885798 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:58.885958 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:58.886799 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.888322 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.890913 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.892316 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.892422 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.892626 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:58.893915 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0810 06:46:58.894077 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15801,"thread_start_us":15662,"threads_started":1}
I20260810 06:46:58.895270 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.866243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:57932)
0810 06:46:58.884573 (+ 18330us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:58.884599 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:58.885397 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:58.886191 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:58.886196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:58.886224 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:58.886620 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.886632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.888559 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.888564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.890729 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.890745 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.893429 (+ 2684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.893440 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.893712 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.893748 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:46:58.893831 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:46:58.894927 (+ 1096us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18212,"mutex_wait_us":4,"thread_start_us":18646,"threads_started":1}
I20260810 06:46:58.896317 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.866611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:37233 (local address 127.0.0.1:33988)
0810 06:46:58.885139 (+ 18528us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:58.885160 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:58.885286 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:58.885925 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:58.885929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:58.885957 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:58.886393 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.886408 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.889245 (+ 2837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.889250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.893546 (+ 4296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.893562 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.895664 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.895670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.895847 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.895878 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:46:58.895946 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 06:46:58.896035 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18406,"mutex_wait_us":46,"thread_start_us":17700,"threads_started":1}
I20260810 06:46:58.896857 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.867184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.10.242.2:37233)
0810 06:46:58.884113 (+ 16929us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:58.884122 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:58.885281 (+ 1159us) server_negotiation.cc:407] Connection header received
0810 06:46:58.885490 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:58.885498 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:58.885732 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:58.885925 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:58.886602 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.889080 (+ 2478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.893982 (+ 4902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.895171 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.895313 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.895501 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:58.896529 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0810 06:46:58.896656 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16777,"thread_start_us":16683,"threads_started":1}
I20260810 06:46:58.897305 3343 tablet_service.cc:1467] Processing CreateTablet for tablet 50a0bf015aec4db4ab483955e9c6abaa (DEFAULT_TABLE table=test-table [id=fd39ec8625774abab782d7ac4f2332a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:58.898511 3343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a0bf015aec4db4ab483955e9c6abaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:58.899662 3098 tablet_service.cc:1467] Processing CreateTablet for tablet 50a0bf015aec4db4ab483955e9c6abaa (DEFAULT_TABLE table=test-table [id=fd39ec8625774abab782d7ac4f2332a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:58.901005 3098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a0bf015aec4db4ab483955e9c6abaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:58.902040 2864 tablet_service.cc:1467] Processing CreateTablet for tablet 50a0bf015aec4db4ab483955e9c6abaa (DEFAULT_TABLE table=test-table [id=fd39ec8625774abab782d7ac4f2332a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:46:58.903357 2864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a0bf015aec4db4ab483955e9c6abaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:46:58.910228 3484 tablet_bootstrap.cc:492] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: Bootstrap starting.
I20260810 06:46:58.913028 3484 tablet_bootstrap.cc:654] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:58.914335 3484 log.cc:824] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:58.916294 3487 tablet_bootstrap.cc:492] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: Bootstrap starting.
I20260810 06:46:58.917178 3484 tablet_bootstrap.cc:492] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: No bootstrap required, opened a new log
I20260810 06:46:58.917748 3484 ts_tablet_manager.cc:1397] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260810 06:46:58.919765 3487 tablet_bootstrap.cc:654] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:58.920511 3488 tablet_bootstrap.cc:492] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Bootstrap starting.
I20260810 06:46:58.920962 3487 log.cc:824] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:58.923681 3488 tablet_bootstrap.cc:654] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Neither blocks nor log segments found. Creating new log.
I20260810 06:46:58.924868 2974 heartbeater.cc:503] Master 127.10.242.62:44249 was elected leader, sending a full tablet report...
I20260810 06:46:58.924870 3488 log.cc:824] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Log is configured to *not* fsync() on all Append() calls
I20260810 06:46:58.924962 3487 tablet_bootstrap.cc:492] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: No bootstrap required, opened a new log
I20260810 06:46:58.925217 3487 ts_tablet_manager.cc:1397] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260810 06:46:58.928254 3484 raft_consensus.cc:348] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:58.928679 3484 raft_consensus.cc:374] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:58.928807 3484 raft_consensus.cc:729] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5111de7999f94e7aaac7a157af082a04, State: Initialized, Role: FOLLOWER
I20260810 06:46:58.929438 3484 consensus_queue.cc:260] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:58.930588 3458 heartbeater.cc:503] Master 127.10.242.62:44249 was elected leader, sending a full tablet report...
I20260810 06:46:58.931180 3484 ts_tablet_manager.cc:1428] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260810 06:46:58.933936 3488 tablet_bootstrap.cc:492] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: No bootstrap required, opened a new log
I20260810 06:46:58.934243 3488 ts_tablet_manager.cc:1397] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.007s
I20260810 06:46:58.937379 3487 raft_consensus.cc:348] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:58.937741 3487 raft_consensus.cc:374] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:58.937834 3487 raft_consensus.cc:729] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263ee39503434734843ace6674894605, State: Initialized, Role: FOLLOWER
I20260810 06:46:58.938369 3487 consensus_queue.cc:260] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:58.939429 3227 heartbeater.cc:503] Master 127.10.242.62:44249 was elected leader, sending a full tablet report...
I20260810 06:46:58.939875 3487 ts_tablet_manager.cc:1428] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: Time spent starting tablet: real 0.015s user 0.011s sys 0.000s
I20260810 06:46:58.942068 3488 raft_consensus.cc:348] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:58.942523 3488 raft_consensus.cc:374] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:46:58.942627 3488 raft_consensus.cc:729] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb594530a1404fda9e6440c0fa297e99, State: Initialized, Role: FOLLOWER
I20260810 06:46:58.943192 3488 consensus_queue.cc:260] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:58.944669 3488 ts_tablet_manager.cc:1428] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Time spent starting tablet: real 0.010s user 0.009s sys 0.001s
I20260810 06:46:58.984481 3501 raft_consensus.cc:482] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:46:58.984774 3501 raft_consensus.cc:504] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:58.986529 3501 leader_election.cc:288] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5111de7999f94e7aaac7a157af082a04 (127.10.242.3:36171), 263ee39503434734843ace6674894605 (127.10.242.2:37233)
I20260810 06:46:58.992939 3475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.986955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:56001 (local address 127.10.242.3:36171)
0810 06:46:58.987182 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:58.987190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:58.987925 (+ 735us) server_negotiation.cc:407] Connection header received
0810 06:46:58.988081 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:58.988085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:58.988153 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:58.988226 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:58.988855 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.989576 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.991293 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.992071 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.992223 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.992294 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:58.992582 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 06:46:58.992702 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:46:58.993026 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.987157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:43871 (local address 127.10.242.2:37233)
0810 06:46:58.987369 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:58.987376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:58.987875 (+ 499us) server_negotiation.cc:407] Connection header received
0810 06:46:58.988052 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:58.988057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:58.988118 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:58.988195 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:46:58.988801 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.989478 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.991285 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.992118 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.992253 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.992321 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:58.992668 (+ 347us) server_negotiation.cc:299] Negotiation successful
0810 06:46:58.992786 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:46:58.993083 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.986968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:36171 (local address 127.10.242.1:56001)
0810 06:46:58.987809 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:58.987831 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:58.987953 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:58.988254 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:58.988258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:58.988281 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:58.988705 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.988715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.989716 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.989719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.991145 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.991157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.992291 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.992297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.992434 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.992460 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:46:58.992539 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:46:58.992672 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":2,"thread_start_us":394,"threads_started":1}
I20260810 06:46:58.993142 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:58.986796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:37233 (local address 127.10.242.1:43871)
0810 06:46:58.987770 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:58.987786 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:46:58.987922 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:46:58.988228 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:46:58.988233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:46:58.988257 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:46:58.988664 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.988677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.989603 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.989607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.991139 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:46:58.991152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:46:58.992397 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:46:58.992402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:46:58.992519 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:58.992541 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:46:58.992617 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 06:46:58.992731 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":19,"thread_start_us":764,"threads_started":1}
I20260810 06:46:58.994223 3382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a0bf015aec4db4ab483955e9c6abaa" candidate_uuid: "cb594530a1404fda9e6440c0fa297e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5111de7999f94e7aaac7a157af082a04" is_pre_election: true
I20260810 06:46:58.994513 3142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a0bf015aec4db4ab483955e9c6abaa" candidate_uuid: "cb594530a1404fda9e6440c0fa297e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ee39503434734843ace6674894605" is_pre_election: true
I20260810 06:46:58.994690 3382 raft_consensus.cc:2454] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb594530a1404fda9e6440c0fa297e99 in term 0.
I20260810 06:46:58.994937 3142 raft_consensus.cc:2454] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb594530a1404fda9e6440c0fa297e99 in term 0.
I20260810 06:46:58.995334 2790 leader_election.cc:302] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5111de7999f94e7aaac7a157af082a04, cb594530a1404fda9e6440c0fa297e99; no voters:
I20260810 06:46:58.995710 3501 raft_consensus.cc:2781] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:46:58.995836 3501 raft_consensus.cc:482] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:46:58.995929 3501 raft_consensus.cc:3037] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:58.998032 3501 raft_consensus.cc:504] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:58.998594 3501 leader_election.cc:288] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [CANDIDATE]: Term 1 election: Requested vote from peers 5111de7999f94e7aaac7a157af082a04 (127.10.242.3:36171), 263ee39503434734843ace6674894605 (127.10.242.2:37233)
I20260810 06:46:58.999020 3382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a0bf015aec4db4ab483955e9c6abaa" candidate_uuid: "cb594530a1404fda9e6440c0fa297e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5111de7999f94e7aaac7a157af082a04"
I20260810 06:46:58.999027 3142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a0bf015aec4db4ab483955e9c6abaa" candidate_uuid: "cb594530a1404fda9e6440c0fa297e99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263ee39503434734843ace6674894605"
I20260810 06:46:58.999184 3142 raft_consensus.cc:3037] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:58.999197 3382 raft_consensus.cc:3037] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:46:59.001286 3142 raft_consensus.cc:2454] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb594530a1404fda9e6440c0fa297e99 in term 1.
I20260810 06:46:59.001351 3382 raft_consensus.cc:2454] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb594530a1404fda9e6440c0fa297e99 in term 1.
I20260810 06:46:59.001679 2786 leader_election.cc:302] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 263ee39503434734843ace6674894605, cb594530a1404fda9e6440c0fa297e99; no voters:
I20260810 06:46:59.001967 3501 raft_consensus.cc:2781] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:46:59.002401 3501 raft_consensus.cc:686] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 1 LEADER]: Becoming Leader. State: Replica: cb594530a1404fda9e6440c0fa297e99, State: Running, Role: LEADER
I20260810 06:46:59.002826 3501 consensus_queue.cc:237] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } }
I20260810 06:46:59.007506 2666 catalog_manager.cc:5495] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb594530a1404fda9e6440c0fa297e99 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "cb594530a1404fda9e6440c0fa297e99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb594530a1404fda9e6440c0fa297e99" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 46059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 } health_report { overall_health: UNKNOWN } } }
I20260810 06:46:59.045687 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.040401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57936 (local address 127.10.242.1:46059)
0810 06:46:59.040593 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:59.040600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:59.040656 (+ 56us) server_negotiation.cc:407] Connection header received
0810 06:46:59.040877 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:59.040881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:59.040972 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:59.041100 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:46:59.041499 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:59.042246 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:59.043570 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:59.043980 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:59.044088 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:59.044563 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:46:59.045058 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:46:59.045187 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:59.045318 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 06:46:59.045399 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:46:59.053959 3382 raft_consensus.cc:1261] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [term 1 FOLLOWER]: Refusing update from remote peer cb594530a1404fda9e6440c0fa297e99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:46:59.053964 3142 raft_consensus.cc:1261] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [term 1 FOLLOWER]: Refusing update from remote peer cb594530a1404fda9e6440c0fa297e99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260810 06:46:59.054669 3459 tablet.cc:2369] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:46:59.055042 3501 consensus_queue.cc:1035] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "263ee39503434734843ace6674894605" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:59.055353 3505 consensus_queue.cc:1035] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5111de7999f94e7aaac7a157af082a04" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:46:59.077955 2665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:59.078296 2665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:46:59.081614 2665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Sending DeleteTablet for 3 replicas of tablet 50a0bf015aec4db4ab483955e9c6abaa
I20260810 06:46:59.082670 3343 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a0bf015aec4db4ab483955e9c6abaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:59 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:39086
I20260810 06:46:59.082702 2864 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a0bf015aec4db4ab483955e9c6abaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:59 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:57932
I20260810 06:46:59.083253 3098 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a0bf015aec4db4ab483955e9c6abaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:46:59 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:33988
I20260810 06:46:59.084120 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 2725
I20260810 06:46:59.097890 3535 tablet_replica.cc:330] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: stopping tablet replica
I20260810 06:46:59.098367 3535 raft_consensus.cc:2229] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:46:59.098417 3532 tablet_replica.cc:330] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: stopping tablet replica
I20260810 06:46:59.098865 3535 raft_consensus.cc:2258] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:59.098834 3532 raft_consensus.cc:2229] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:46:59.099493 3532 raft_consensus.cc:2258] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:59.100994 3535 ts_tablet_manager.cc:1905] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:59.101416 3532 ts_tablet_manager.cc:1905] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:59.103940 3536 tablet_replica.cc:330] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: stopping tablet replica
I20260810 06:46:59.104342 3536 raft_consensus.cc:2229] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:46:59.104854 3536 raft_consensus.cc:2258] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:46:59.106866 3536 ts_tablet_manager.cc:1905] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:46:59.108435 3535 ts_tablet_manager.cc:1918] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:59.108665 3535 log.cc:1196] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/50a0bf015aec4db4ab483955e9c6abaa
I20260810 06:46:59.109300 3535 ts_tablet_manager.cc:1939] T 50a0bf015aec4db4ab483955e9c6abaa P 5111de7999f94e7aaac7a157af082a04: Deleting consensus metadata
I20260810 06:46:59.111264 2636 catalog_manager.cc:4841] TS 5111de7999f94e7aaac7a157af082a04 (127.10.242.3:36171): tablet 50a0bf015aec4db4ab483955e9c6abaa (table test-table [id=fd39ec8625774abab782d7ac4f2332a0]) successfully deleted
I20260810 06:46:59.113550 3536 ts_tablet_manager.cc:1918] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:59.113718 3536 log.cc:1196] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/50a0bf015aec4db4ab483955e9c6abaa
I20260810 06:46:59.114297 3536 ts_tablet_manager.cc:1939] T 50a0bf015aec4db4ab483955e9c6abaa P 263ee39503434734843ace6674894605: Deleting consensus metadata
I20260810 06:46:59.115109 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.085645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.10.242.1:46059)
0810 06:46:59.085812 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:59.085818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:59.085835 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:46:59.085896 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:59.085902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:59.085977 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:59.086069 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:59.088094 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:59.089377 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:59.091386 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:59.092117 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:59.092231 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:59.106397 (+ 14166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:59.106412 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:59.106428 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:59.106481 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:59.110438 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:59.111352 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:59.111361 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:59.111365 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:59.111536 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:59.112007 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:59.112016 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:59.112020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:59.112539 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:59.112890 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:59.114441 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0810 06:46:59.114705 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:46:59.116122 2965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:59.116333 2635 catalog_manager.cc:4841] TS 263ee39503434734843ace6674894605 (127.10.242.2:37233): tablet 50a0bf015aec4db4ab483955e9c6abaa (table test-table [id=fd39ec8625774abab782d7ac4f2332a0]) successfully deleted
I20260810 06:46:59.212477 3532 ts_tablet_manager.cc:1918] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:46:59.212805 3532 log.cc:1196] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/50a0bf015aec4db4ab483955e9c6abaa
I20260810 06:46:59.213281 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 2725
W20260810 06:46:59.245981 2643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.1:46059 (error 108)
I20260810 06:46:59.246127 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 2980
I20260810 06:46:59.247862 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.247402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:57948)
0810 06:46:59.247573 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:59.247709 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:46059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 06:46:59.248100 2643 catalog_manager.cc:4584] TS cb594530a1404fda9e6440c0fa297e99 (127.10.242.1:46059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50a0bf015aec4db4ab483955e9c6abaa: Network error: Client connection negotiation failed: client connection to 127.10.242.1:46059: connect: Connection refused (error 111)
I20260810 06:46:59.259455 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.246925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.10.242.2:37233)
0810 06:46:59.247142 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:59.247149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:59.247171 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:46:59.247334 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:59.247338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:59.247421 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:59.247535 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:59.249087 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:59.249823 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:59.251512 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:59.251911 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:59.252003 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:59.254501 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:59.254513 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:59.254529 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:59.254563 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:59.257086 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:59.257759 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:59.257765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:59.257768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:59.257886 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:59.258219 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:59.258223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:59.258224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:59.258578 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:59.258792 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:59.259024 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 06:46:59.259190 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:46:59.260126 3221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:59.301891 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.301320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:57954)
0810 06:46:59.301569 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:59.301693 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:46059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 06:46:59.303103 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.302644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:57960)
0810 06:46:59.302819 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:59.302931 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:46059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:46:59.363829 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.363263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:57970)
0810 06:46:59.363506 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:59.363619 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:46059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 06:46:59.365079 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 2980
I20260810 06:46:59.365377 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.364913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:57974)
0810 06:46:59.365150 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:59.365239 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:46059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260810 06:46:59.385067 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 3231
I20260810 06:46:59.398464 3475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.385811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39094 (local address 127.10.242.3:36171)
0810 06:46:59.385988 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:46:59.385994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:46:59.386015 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:46:59.386255 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:46:59.386260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:46:59.386331 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:46:59.386417 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:46:59.387785 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:59.388507 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:59.389791 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:46:59.390262 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:46:59.390371 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:46:59.392858 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:46:59.392873 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:46:59.392890 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:46:59.392927 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:46:59.395735 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:59.396472 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:59.396480 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:59.396482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:59.396620 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:46:59.396974 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:46:59.396978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:46:59.396980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:46:59.397331 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:46:59.397613 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 06:46:59.397923 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 06:46:59.398131 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:46:59.399022 3446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:59.449966 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.449499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:57988)
0810 06:46:59.449716 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:59.449808 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:46059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:46:59.450996 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.450610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:57994)
0810 06:46:59.450803 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:59.450876 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:46059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 06:46:59.488557 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 3231
I20260810 06:46:59.513490 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 2602
I20260810 06:46:59.514643 2716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:46:59.625751 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:46:59.625115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:46059 (local address 127.0.0.1:58000)
0810 06:46:59.625389 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 06:46:59.625502 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:46059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260810 06:46:59.625833 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 2602
I20260810 06:46:59.650552 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:44249
--webserver_interface=127.10.242.62
--webserver_port=40279
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:44249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:46:59.976560 3544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:46:59.976838 3544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:46:59.976964 3544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:46:59.983966 3544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:46:59.984071 3544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:46:59.984146 3544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:46:59.984215 3544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:46:59.984272 3544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:46:59.993754 3544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:44249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:44249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=40279
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:46:59.994995 3544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:46:59.996276 3544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:00.007324 3552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:00.007390 3551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:00.007359 3555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:00.008154 3544 server_base.cc:1034] running on GCE node
I20260810 06:47:00.008855 3544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:00.010109 3544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:00.011351 3544 hybrid_clock.cc:648] HybridClock initialized: now 1786344420011293 us; error 45 us; skew 500 ppm
I20260810 06:47:00.011682 3544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:00 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:00.014847 3544 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:47:00.016384 3544 webserver.cc:460] Webserver started at http://127.10.242.62:40279/ using document root <none> and password file <none>
I20260810 06:47:00.017228 3544 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:00.017365 3544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:00.023799 3544 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:47:00.026619 3568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:00.027863 3544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:47:00.027995 3544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "6d6866c2413b4331a25a7d5e368dfb33"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "0a4bde9a2fa89e8f1a1a086048e26155"
server_key_iv: "d0485eb1ab6eea4407bec8b5c5c93427"
server_key_version: "encryptionkey@0"
I20260810 06:47:00.028900 3544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:00.048110 3544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:00.051751 3544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:00.051975 3544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:00.069674 3544 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:44249
I20260810 06:47:00.069694 3654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:44249 every 8 connection(s)
I20260810 06:47:00.071139 3544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:00.073465 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 3544
I20260810 06:47:00.074307 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:46059
--local_ip_for_outbound_sockets=127.10.242.1
--tserver_master_addrs=127.10.242.62:44249
--webserver_port=43897
--webserver_interface=127.10.242.1
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:00.077967 3655 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:47:00.081115 3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Bootstrap starting.
I20260810 06:47:00.101101 3655 log.cc:824] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:00.117542 3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:47:00.118055 3655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33: Bootstrap complete.
I20260810 06:47:00.127197 3655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:47:00.128108 3655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d6866c2413b4331a25a7d5e368dfb33, State: Initialized, Role: FOLLOWER
I20260810 06:47:00.128696 3655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:47:00.128923 3655 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:47:00.129033 3655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:47:00.129208 3655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:47:00.133198 3655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:47:00.133720 3655 leader_election.cc:302] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d6866c2413b4331a25a7d5e368dfb33; no voters:
I20260810 06:47:00.134379 3655 leader_election.cc:288] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:47:00.134517 3660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:47:00.136518 3655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:00.141179 3660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [term 3 LEADER]: Becoming Leader. State: Replica: 6d6866c2413b4331a25a7d5e368dfb33, State: Running, Role: LEADER
I20260810 06:47:00.141803 3660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } }
I20260810 06:47:00.144440 3661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6d6866c2413b4331a25a7d5e368dfb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } } }
I20260810 06:47:00.144804 3661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:00.145361 3662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d6866c2413b4331a25a7d5e368dfb33. Latest consensus state: current_term: 3 leader_uuid: "6d6866c2413b4331a25a7d5e368dfb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6866c2413b4331a25a7d5e368dfb33" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 44249 } } }
I20260810 06:47:00.145653 3662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6866c2413b4331a25a7d5e368dfb33 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:00.152240 3676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:00.160841 3676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:47:00.162386 3676 catalog_manager.cc:1259] Loaded cluster ID: 8c6a912fbe7b40bb9a6219a4d5a1e1a3
I20260810 06:47:00.162554 3676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 06:47:00.164781 3676 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd7b6d569ad google::LogMessage::Fail() at ??:0
@ 0x7fd7b6d5988a google::LogMessage::SendToLog() at ??:0
@ 0x7fd7b6d564de google::LogMessage::Flush() at ??:0
@ 0x7fd7b6d57a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd7cc789512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd7cc786976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd7cc80e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd7cc80e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd7b99a2345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd7b99a591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd7b99a5799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd7b9980eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd7bcc5a6db start_thread at ??:0
@ 0x7fd7b5bf061f clone at ??:0
W20260810 06:47:00.495532 3658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:00.495826 3658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:00.495939 3658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:00.503094 3658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:00.503201 3658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:00.503320 3658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:47:00.512784 3658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:46059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=43897
--tserver_master_addrs=127.10.242.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:00.514063 3658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:00.515367 3658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:00.528339 3715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:00.528431 3712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:00.528388 3713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:00.529503 3658 server_base.cc:1034] running on GCE node
I20260810 06:47:00.530117 3658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:00.531047 3658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:00.532246 3658 hybrid_clock.cc:648] HybridClock initialized: now 1786344420532208 us; error 41 us; skew 500 ppm
I20260810 06:47:00.532538 3658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:00 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:00.535591 3658 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:47:00.537011 3658 webserver.cc:460] Webserver started at http://127.10.242.1:43897/ using document root <none> and password file <none>
I20260810 06:47:00.537688 3658 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:00.537810 3658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:00.543057 3658 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:47:00.545689 3728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:00.547015 3658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:47:00.547180 3658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "cb594530a1404fda9e6440c0fa297e99"
format_stamp: "Formatted at 2026-08-10 06:46:54 on dist-test-slave-zj8q"
server_key: "f21186b85906e89f41e33e495ef7453e"
server_key_iv: "85434ff2335cea78633b3954b1546a9e"
server_key_version: "encryptionkey@0"
I20260810 06:47:00.548158 3658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:00.569905 3658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:00.573750 3658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:00.574011 3658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:00.575026 3658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:00.576772 3744 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260810 06:47:00.581954 3744 ts_tablet_manager.cc:1852] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260810 06:47:00.582964 3744 ts_tablet_manager.cc:1905] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:00.590080 3744 ts_tablet_manager.cc:1918] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:00.590337 3744 ts_tablet_manager.cc:1939] T 50a0bf015aec4db4ab483955e9c6abaa P cb594530a1404fda9e6440c0fa297e99: Deleting consensus metadata
I20260810 06:47:00.590698 3658 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260810 06:47:00.590813 3658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.014s user 0.001s sys 0.000s
I20260810 06:47:00.590972 3658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:00.591063 3658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:00.622561 3658 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:46059
I20260810 06:47:00.622628 3935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:46059 every 8 connection(s)
I20260810 06:47:00.623852 3658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:47:00.625758 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 3658
I20260810 06:47:00.626593 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:37233
--local_ip_for_outbound_sockets=127.10.242.2
--tserver_master_addrs=127.10.242.62:44249
--webserver_port=41597
--webserver_interface=127.10.242.2
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:00.629261 3938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:00.627344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:35569)
0810 06:47:00.628163 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:00.628405 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":337,"threads_started":1}
W20260810 06:47:00.629957 3936 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:44249 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.62:44249: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
I20260810 06:47:00.630805 3938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:00.630471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:34923)
0810 06:47:00.630622 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:00.630696 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:47:00.631690 3938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:00.631377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:35425)
0810 06:47:00.631492 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:00.631566 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:47:00.631894 3936 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:47:00.960120 3939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:00.960394 3939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:00.960489 3939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:00.967613 3939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:00.967720 3939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:00.967839 3939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:47:00.977409 3939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:37233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=41597
--tserver_master_addrs=127.10.242.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:00.978659 3939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:00.979964 3939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:00.992595 3947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:00.992552 3948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:00.992668 3950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:00.993427 3939 server_base.cc:1034] running on GCE node
I20260810 06:47:00.994005 3939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:00.994915 3939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:00.996105 3939 hybrid_clock.cc:648] HybridClock initialized: now 1786344420996046 us; error 51 us; skew 500 ppm
I20260810 06:47:00.996403 3939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:00 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344420, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:00.999400 3939 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:47:01.000708 3939 webserver.cc:460] Webserver started at http://127.10.242.2:41597/ using document root <none> and password file <none>
I20260810 06:47:01.001346 3939 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:01.001449 3939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:01.006578 3939 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:47:01.009192 3960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:01.010473 3939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:47:01.010609 3939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "263ee39503434734843ace6674894605"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "d4af9537a8b2aec0a823f8c96e70e07a"
server_key_iv: "5a7a4f409ba91d48d49d471ff3ee40d8"
server_key_version: "encryptionkey@0"
I20260810 06:47:01.011564 3939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:01.028090 3939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:01.032181 3939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:01.032419 3939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:01.033391 3939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:01.034722 3939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:01.034821 3939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:01.034906 3939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:01.034962 3939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:01.065503 3939 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:37233
I20260810 06:47:01.065577 4161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:37233 every 8 connection(s)
I20260810 06:47:01.066988 3939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:47:01.071532 4166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:01.069775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:56385)
0810 06:47:01.070499 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:01.070703 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":286,"threads_started":1}
W20260810 06:47:01.072196 4162 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:44249 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.62:44249: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
I20260810 06:47:01.072926 4166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:01.072626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:48105)
0810 06:47:01.072758 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:01.072823 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:47:01.073716 4166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:01.073411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:60655)
0810 06:47:01.073538 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:01.073608 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:47:01.073923 4162 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:47:01.075006 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 3939
I20260810 06:47:01.075630 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:36171
--local_ip_for_outbound_sockets=127.10.242.3
--tserver_master_addrs=127.10.242.62:44249
--webserver_port=46329
--webserver_interface=127.10.242.3
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:01.402891 4167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:01.403146 4167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:01.403277 4167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:01.410315 4167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:01.410409 4167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:01.410583 4167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:47:01.419801 4167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:36171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=46329
--tserver_master_addrs=127.10.242.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:01.420989 4167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:01.422271 4167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:01.434660 4177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:01.434700 4179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:01.434764 4181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:01.435833 4167 server_base.cc:1034] running on GCE node
I20260810 06:47:01.436354 4167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:01.437150 4167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:01.438338 4167 hybrid_clock.cc:648] HybridClock initialized: now 1786344421438287 us; error 42 us; skew 500 ppm
I20260810 06:47:01.438621 4167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:01 dist-test-slave-zj8q krb5kdc[1647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344421, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:01.441574 4167 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:47:01.442889 4167 webserver.cc:460] Webserver started at http://127.10.242.3:46329/ using document root <none> and password file <none>
I20260810 06:47:01.443511 4167 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:01.443632 4167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:01.448587 4167 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 06:47:01.451128 4192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:01.452405 4167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:47:01.452551 4167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "5111de7999f94e7aaac7a157af082a04"
format_stamp: "Formatted at 2026-08-10 06:46:55 on dist-test-slave-zj8q"
server_key: "e03197be1cbb7bdf3c7bdf34c5c5d98c"
server_key_iv: "cc8e957981c502cf8c2ec99958861931"
server_key_version: "encryptionkey@0"
I20260810 06:47:01.453519 4167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:01.466419 4167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:01.470285 4167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:01.470530 4167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:01.471397 4167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:01.472625 4167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:01.472726 4167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:01.472831 4167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:01.472901 4167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:01.502508 4167 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:36171
I20260810 06:47:01.502571 4392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:36171 every 8 connection(s)
I20260810 06:47:01.503854 4167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:47:01.508298 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:01.506551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:46347)
0810 06:47:01.507281 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:01.507480 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":268,"threads_started":1}
W20260810 06:47:01.508970 4398 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:44249 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.62:44249: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
I20260810 06:47:01.509685 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:01.509392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:57085)
0810 06:47:01.509514 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:01.509582 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 06:47:01.510422 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:01.510113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:50507)
0810 06:47:01.510241 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:01.510312 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 06:47:01.510643 4398 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:47:01.511724 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 4167
I20260810 06:47:01.633291 4409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:01.632420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:54543)
0810 06:47:01.632941 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:01.633044 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":243,"threads_started":1}
I20260810 06:47:02.075291 4411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:02.074487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:37451)
0810 06:47:02.074950 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:02.075065 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":215,"threads_started":1}
I20260810 06:47:02.511891 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:02.511108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:35355)
0810 06:47:02.511553 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:02.511668 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":224,"threads_started":1}
I20260810 06:47:02.634949 4414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:02.634112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:60701)
0810 06:47:02.634668 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:02.634775 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":281,"threads_started":1}
I20260810 06:47:03.076750 4415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:03.076098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:57645)
0810 06:47:03.076517 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:03.076594 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":203,"threads_started":1}
I20260810 06:47:03.513367 4416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:03.512672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:34865)
0810 06:47:03.513116 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:03.513207 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":219,"threads_started":1}
I20260810 06:47:03.636467 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:03.635771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:34805)
0810 06:47:03.636221 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:03.636310 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":210,"threads_started":1}
I20260810 06:47:04.078544 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:04.077652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:39517)
0810 06:47:04.078213 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:04.078320 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":284,"threads_started":1}
I20260810 06:47:04.514905 4421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:04.514145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:58547)
0810 06:47:04.514663 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:04.514741 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":275,"threads_started":1}
I20260810 06:47:04.638072 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:04.637341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:51355)
0810 06:47:04.637796 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:04.637879 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":222,"threads_started":1}
I20260810 06:47:05.080129 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:05.079372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:56485)
0810 06:47:05.079852 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:05.079936 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":253,"threads_started":1}
I20260810 06:47:05.516400 4425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:05.515715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:39877)
0810 06:47:05.516136 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:05.516214 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":201,"threads_started":1}
I20260810 06:47:05.639660 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:05.638910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:60637)
0810 06:47:05.639385 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:05.639468 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":233,"threads_started":1}
I20260810 06:47:06.081717 4428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:06.080959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:44761)
0810 06:47:06.081455 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:06.081536 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":268,"threads_started":1}
I20260810 06:47:06.517964 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:06.517238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:48321)
0810 06:47:06.517712 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:06.517794 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":236,"threads_started":1}
I20260810 06:47:06.641203 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:06.640478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:35355)
0810 06:47:06.640919 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:06.641004 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":218,"threads_started":1}
I20260810 06:47:07.083249 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:07.082547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:50253)
0810 06:47:07.082983 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:07.083074 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":223,"threads_started":1}
I20260810 06:47:07.519517 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:07.518828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:38177)
0810 06:47:07.519266 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:07.519354 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":215,"threads_started":1}
I20260810 06:47:07.642860 4443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:07.641993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:60277)
0810 06:47:07.642553 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:07.642659 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":264,"threads_started":1}
I20260810 06:47:08.084877 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:08.084104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:50679)
0810 06:47:08.084576 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:08.084663 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":202,"threads_started":1}
I20260810 06:47:08.521071 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:08.520312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:42441)
0810 06:47:08.520776 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:08.520851 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":203,"threads_started":1}
I20260810 06:47:08.644433 4449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:08.643689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:44855)
0810 06:47:08.644165 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:08.644250 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":240,"threads_started":1}
I20260810 06:47:09.086498 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:09.085745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:37803)
0810 06:47:09.086231 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:09.086321 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":238,"threads_started":1}
I20260810 06:47:09.522631 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:09.521915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:59267)
0810 06:47:09.522379 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:09.522464 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":255,"threads_started":1}
I20260810 06:47:09.646056 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:09.645317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:42165)
0810 06:47:09.645814 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:09.645903 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":248,"threads_started":1}
I20260810 06:47:10.088084 4453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:10.087314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:47615)
0810 06:47:10.087828 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:10.087911 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":235,"threads_started":1}
I20260810 06:47:10.524158 4454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:10.523441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:37347)
0810 06:47:10.523907 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:10.523987 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":241,"threads_started":1}
I20260810 06:47:10.647600 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:10.646899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:35091)
0810 06:47:10.647336 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:10.647420 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":222,"threads_started":1}
I20260810 06:47:11.089694 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:11.088962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:40849)
0810 06:47:11.089445 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:11.089530 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":261,"threads_started":1}
I20260810 06:47:11.525928 4459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:11.525176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:39707)
0810 06:47:11.525664 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:11.525747 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":253,"threads_started":1}
I20260810 06:47:11.649183 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:11.648397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:41279)
0810 06:47:11.648921 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:11.649002 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":321,"threads_started":1}
I20260810 06:47:12.091393 4461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:12.090608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:54969)
0810 06:47:12.091095 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:12.091209 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":241,"threads_started":1}
I20260810 06:47:12.527464 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:12.526771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:33313)
0810 06:47:12.527190 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:12.527281 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":214,"threads_started":1}
I20260810 06:47:12.650821 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:12.649992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:50693)
0810 06:47:12.650473 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:12.650579 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":261,"threads_started":1}
I20260810 06:47:13.092967 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:13.092298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:46245)
0810 06:47:13.092717 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:13.092803 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":208,"threads_started":1}
I20260810 06:47:13.528891 4467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:13.528251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:50129)
0810 06:47:13.528659 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:13.528738 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":205,"threads_started":1}
I20260810 06:47:13.652429 4468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:13.651630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:33967)
0810 06:47:13.652113 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:13.652248 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":224,"threads_started":1}
I20260810 06:47:14.094619 4469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:14.093858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:35289)
0810 06:47:14.094343 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:14.094432 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":225,"threads_started":1}
I20260810 06:47:14.530407 4470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:14.529700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:39437)
0810 06:47:14.530127 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:14.530228 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":210,"threads_started":1}
I20260810 06:47:14.653967 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:14.653239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:44703)
0810 06:47:14.653686 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:14.653773 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":212,"threads_started":1}
I20260810 06:47:15.096109 4473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:15.095390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:43581)
0810 06:47:15.095831 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:15.095913 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":213,"threads_started":1}
I20260810 06:47:15.531847 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:15.531179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:60575)
0810 06:47:15.531586 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:15.531667 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":202,"threads_started":1}
I20260810 06:47:15.655678 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:15.654927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.1:47051)
0810 06:47:15.655384 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:15.655491 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":232,"threads_started":1}
I20260810 06:47:16.097935 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:16.097185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.2:59831)
0810 06:47:16.097650 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:16.097742 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":229,"threads_started":1}
I20260810 06:47:16.516332 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 3658
I20260810 06:47:16.535849 4479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:16.535023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:44249 (local address 127.10.242.3:45363)
0810 06:47:16.535484 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:16.535600 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:44249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":229,"threads_started":1}
I20260810 06:47:16.544556 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:16.517497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.10.242.1:46059)
0810 06:47:16.518078 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:16.518096 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:16.518144 (+ 48us) server_negotiation.cc:407] Connection header received
0810 06:47:16.519757 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:16.519766 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:16.520196 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:16.520522 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:16.522009 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:16.523292 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:16.524543 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:16.525217 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:16.525394 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:16.536065 (+ 10671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:16.536075 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:16.536099 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:16.536150 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:16.540488 (+ 4338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:16.541399 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:16.541409 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:16.541418 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:16.541640 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:16.542123 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:16.542128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:16.542132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:16.542938 (+ 806us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:16.543245 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:16.543933 (+ 688us) server_negotiation.cc:299] Negotiation successful
0810 06:47:16.544268 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":233,"threads_started":1}
I20260810 06:47:16.546725 3928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:16.643078 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 3658
I20260810 06:47:16.666678 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 3939
I20260810 06:47:16.683235 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:16.667689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51172 (local address 127.10.242.2:37233)
0810 06:47:16.668296 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:16.668311 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:16.668350 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:47:16.669935 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:16.669945 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:16.670438 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:16.670721 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:16.671995 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:16.673225 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:16.674500 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:16.675162 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:16.675292 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:16.677696 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:16.677703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:16.677718 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:16.677752 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:16.680512 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:16.681062 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:16.681067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:16.681073 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:16.681257 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:16.681531 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:16.681534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:16.681536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:16.681990 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:16.682267 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:16.682567 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 06:47:16.682941 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":301,"threads_started":1}
I20260810 06:47:16.684645 4152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:16.766790 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 3939
I20260810 06:47:16.792296 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 4167
I20260810 06:47:16.810091 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:16.793357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57342 (local address 127.10.242.3:36171)
0810 06:47:16.793991 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:16.794006 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:16.794047 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:47:16.795935 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:16.795945 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:16.796378 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:16.796695 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:16.798265 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:16.799470 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:16.800773 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:16.801420 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:16.801606 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:16.804306 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:16.804314 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:16.804329 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:16.804377 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:16.807002 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:16.807653 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:16.807661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:16.807667 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:16.807825 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:16.808191 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:16.808194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:16.808196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:16.808692 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:16.808989 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:16.809418 (+ 429us) server_negotiation.cc:299] Negotiation successful
0810 06:47:16.809795 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":313,"threads_started":1}
I20260810 06:47:16.811389 4385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:16.893332 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 4167
2026-08-10T06:47:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24627 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:47:16 dist-test-slave-zj8q krb5kdc[4491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:47:16 dist-test-slave-zj8q krb5kdc[4491](info): set up 2 sockets
Aug 10 06:47:16 dist-test-slave-zj8q krb5kdc[4491](info): commencing operation
krb5kdc: starting...
W20260810 06:47:18.124635 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:47:18 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:47:18.395315 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40091
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:40091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:18.719931 4508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:18.720201 4508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:18.720328 4508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:18.727396 4508 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:47:18.727485 4508 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:47:18.727545 4508 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:47:18.727626 4508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:18.727697 4508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:18.727792 4508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:18.727854 4508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:18.727929 4508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:18.737529 4508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:40091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:18.738765 4508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:18.740041 4508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:18.751274 4518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:18.751319 4519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:18.751355 4522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:18.752137 4508 server_base.cc:1034] running on GCE node
I20260810 06:47:18.752833 4508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:18.754062 4508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:18.755277 4508 hybrid_clock.cc:648] HybridClock initialized: now 1786344438755218 us; error 28 us; skew 500 ppm
I20260810 06:47:18.755568 4508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:18 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344438, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:18.758505 4508 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:47:18.759778 4508 webserver.cc:460] Webserver started at http://127.10.242.62:40971/ using document root <none> and password file <none>
I20260810 06:47:18.760416 4508 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:18.760511 4508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:18.760807 4508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:18.763722 4508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "5eb0b721873049109f85696f031acb11"
format_stamp: "Formatted at 2026-08-10 06:47:18 on dist-test-slave-zj8q"
server_key: "3f8b4040ad05d54d6218f90f6b0c74c3"
server_key_iv: "5b16cb6cddacb4fa7c79cab2ef97fbaf"
server_key_version: "encryptionkey@0"
I20260810 06:47:18.764456 4508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "5eb0b721873049109f85696f031acb11"
format_stamp: "Formatted at 2026-08-10 06:47:18 on dist-test-slave-zj8q"
server_key: "3f8b4040ad05d54d6218f90f6b0c74c3"
server_key_iv: "5b16cb6cddacb4fa7c79cab2ef97fbaf"
server_key_version: "encryptionkey@0"
I20260810 06:47:18.769284 4508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 06:47:18.772569 4530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:18.773779 4508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:47:18.773934 4508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "5eb0b721873049109f85696f031acb11"
format_stamp: "Formatted at 2026-08-10 06:47:18 on dist-test-slave-zj8q"
server_key: "3f8b4040ad05d54d6218f90f6b0c74c3"
server_key_iv: "5b16cb6cddacb4fa7c79cab2ef97fbaf"
server_key_version: "encryptionkey@0"
I20260810 06:47:18.774142 4508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:18.791175 4508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:18.794724 4508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:18.794967 4508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:18.811919 4508 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:40091
I20260810 06:47:18.811952 4618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:40091 every 8 connection(s)
I20260810 06:47:18.813120 4508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:18.816536 4620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:18.817698 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 4508
I20260810 06:47:18.817936 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:47:18.818778 11208 external_mini_cluster.cc:1428] Setting key 15a16a6a872fff674832d32541265ee9
I20260810 06:47:18.830915 4620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Bootstrap starting.
Aug 10 06:47:18 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:47:18.834728 4620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:18.835809 4620 log.cc:824] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:18.838856 4620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: No bootstrap required, opened a new log
I20260810 06:47:18.842195 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:18.820803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.10.242.62:40091)
0810 06:47:18.821627 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:18.821638 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:18.821688 (+ 50us) server_negotiation.cc:407] Connection header received
0810 06:47:18.823327 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:18.823349 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:18.823734 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:18.824053 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:18.825908 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:18.827244 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:18.829140 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:18.829808 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:18.829992 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:18.835020 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:18.835028 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:18.835042 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:18.835089 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:18.838108 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:18.838956 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:18.838964 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:18.838972 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:18.839157 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:18.839588 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:18.839591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:18.839593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:18.840097 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:18.840311 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:18.840829 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 06:47:18.841179 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":418,"threads_started":1}
I20260810 06:47:18.846772 4620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:47:18.847107 4620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:18.847206 4620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb0b721873049109f85696f031acb11, State: Initialized, Role: FOLLOWER
I20260810 06:47:18.847712 4620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:47:18.847926 4620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:47:18.848064 4620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:47:18.848215 4620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:18.850309 4620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:47:18.850773 4620 leader_election.cc:302] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eb0b721873049109f85696f031acb11; no voters:
I20260810 06:47:18.851310 4620 leader_election.cc:288] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:47:18.851428 4628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:18.852046 4628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 1 LEADER]: Becoming Leader. State: Replica: 5eb0b721873049109f85696f031acb11, State: Running, Role: LEADER
I20260810 06:47:18.852521 4628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:47:18.853130 4620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:18.855168 4629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5eb0b721873049109f85696f031acb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } } }
I20260810 06:47:18.855517 4629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:18.856213 4630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eb0b721873049109f85696f031acb11. Latest consensus state: current_term: 1 leader_uuid: "5eb0b721873049109f85696f031acb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } } }
I20260810 06:47:18.856585 4630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:18.858899 4638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:18.863894 4638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:47:18.872021 4638 catalog_manager.cc:1347] Generated new cluster ID: d91b165ff50b4ac5a67a2e62b17d3901
I20260810 06:47:18.872120 4638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:47:18.893811 4638 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:47:18.895591 4638 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:47:18.905824 4638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Generated new TSK 0
I20260810 06:47:18.906818 4638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:47:18.975205 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40091
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:19.304415 4664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:19.304700 4664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:19.304843 4664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:19.312057 4664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:19.312186 4664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:19.312345 4664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:47:19.321841 4664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:19.323123 4664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:19.324417 4664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:19.336901 4671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:19.336964 4672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:19.336992 4676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:19.337683 4664 server_base.cc:1034] running on GCE node
I20260810 06:47:19.338332 4664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:19.339183 4664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:19.340412 4664 hybrid_clock.cc:648] HybridClock initialized: now 1786344439340369 us; error 26 us; skew 500 ppm
I20260810 06:47:19.340716 4664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:19 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344439, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:19.343781 4664 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:47:19.345077 4664 webserver.cc:460] Webserver started at http://127.10.242.1:39879/ using document root <none> and password file <none>
I20260810 06:47:19.345726 4664 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:19.345846 4664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:19.346134 4664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:19.348937 4664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "721bc42effaf4bc3a2a3702d9504b229"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "938f595fb4f9154c6313065b9b035b3e"
server_key_iv: "9e0699b3089ff063ff64e72cb5bfd241"
server_key_version: "encryptionkey@0"
I20260810 06:47:19.349648 4664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "721bc42effaf4bc3a2a3702d9504b229"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "938f595fb4f9154c6313065b9b035b3e"
server_key_iv: "9e0699b3089ff063ff64e72cb5bfd241"
server_key_version: "encryptionkey@0"
I20260810 06:47:19.354576 4664 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 06:47:19.357755 4686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:19.358999 4664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:47:19.359140 4664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "721bc42effaf4bc3a2a3702d9504b229"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "938f595fb4f9154c6313065b9b035b3e"
server_key_iv: "9e0699b3089ff063ff64e72cb5bfd241"
server_key_version: "encryptionkey@0"
I20260810 06:47:19.359319 4664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:19.386106 4664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:19.393615 4664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:19.393940 4664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:19.395000 4664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:19.396476 4664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:19.396575 4664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:19.396706 4664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:19.396770 4664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:19.437495 4664 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:37171
I20260810 06:47:19.437496 4881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:37171 every 8 connection(s)
I20260810 06:47:19.439277 4664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:47:19.443653 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 4664
I20260810 06:47:19.443876 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:47:19.444706 11208 external_mini_cluster.cc:1428] Setting key b9a573759ed33f6649392c71b1297114
Aug 10 06:47:19 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344439, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:47:19.460242 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:19.441992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:33867 (local address 127.10.242.62:40091)
0810 06:47:19.442456 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:19.442460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:19.446289 (+ 3829us) server_negotiation.cc:407] Connection header received
0810 06:47:19.446352 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:19.446357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:19.446425 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:19.446532 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:19.448166 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:19.448881 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:19.450489 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:19.450902 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:19.451019 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:19.454432 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:19.454438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:19.454442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:19.454479 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:19.457332 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:19.458010 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:19.458015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:19.458018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:19.458186 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:19.458618 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:19.458621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:19.458623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:19.458869 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:19.458976 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:19.459769 (+ 793us) server_negotiation.cc:299] Negotiation successful
0810 06:47:19.459958 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":248,"threads_started":1}
I20260810 06:47:19.461476 4888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:19.442701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:33867)
0810 06:47:19.443568 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:19.443617 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:19.445913 (+ 2296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:19.446839 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:19.446852 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:19.447417 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:19.447978 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:19.447996 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:19.449049 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:19.449054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:19.450345 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:19.450363 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:19.451121 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:19.451129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:19.451371 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:19.451970 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:19.451992 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:19.454134 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:19.457457 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:19.457463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:19.457473 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:19.457882 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:19.458280 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:19.458285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:19.458288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:19.458523 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:19.458981 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:19.458989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:19.459385 (+ 396us) client_negotiation.cc:769] Sending connection context
0810 06:47:19.459674 (+ 289us) client_negotiation.cc:240] Negotiation successful
0810 06:47:19.460156 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":326,"threads_started":1}
I20260810 06:47:19.463024 4882 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40091
I20260810 06:47:19.463357 4882 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:19.464020 4882 heartbeater.cc:511] Master 127.10.242.62:40091 requested a full tablet report, sending...
I20260810 06:47:19.466030 4558 ts_manager.cc:194] Registered new tserver with Master: 721bc42effaf4bc3a2a3702d9504b229 (127.10.242.1:37171)
I20260810 06:47:19.467454 4558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:33867
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:47:19.503347 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40091
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:19.831965 4892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:19.832263 4892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:19.832397 4892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:19.839665 4892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:19.839771 4892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:19.839919 4892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:47:19.849604 4892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:19.850867 4892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:19.852164 4892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:19.865012 4898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:19.865041 4903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:19.864985 4901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:19.865777 4892 server_base.cc:1034] running on GCE node
I20260810 06:47:19.866405 4892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:19.867290 4892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:19.868497 4892 hybrid_clock.cc:648] HybridClock initialized: now 1786344439868428 us; error 48 us; skew 500 ppm
I20260810 06:47:19.868810 4892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:19 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344439, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:19.871857 4892 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:47:19.873172 4892 webserver.cc:460] Webserver started at http://127.10.242.2:43009/ using document root <none> and password file <none>
I20260810 06:47:19.873793 4892 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:19.873891 4892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:19.874145 4892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:19.877513 4892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "c6599f4a1d0347a199fa661abf0ac14b"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "204545b00ded816f6e120921bb9c2a0d"
server_key_iv: "60c886d4448a77f3a3fe9083b69b6cd3"
server_key_version: "encryptionkey@0"
I20260810 06:47:19.878561 4892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "c6599f4a1d0347a199fa661abf0ac14b"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "204545b00ded816f6e120921bb9c2a0d"
server_key_iv: "60c886d4448a77f3a3fe9083b69b6cd3"
server_key_version: "encryptionkey@0"
I20260810 06:47:19.883610 4892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:47:19.886916 4916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:19.888121 4892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:47:19.888248 4892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "c6599f4a1d0347a199fa661abf0ac14b"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "204545b00ded816f6e120921bb9c2a0d"
server_key_iv: "60c886d4448a77f3a3fe9083b69b6cd3"
server_key_version: "encryptionkey@0"
I20260810 06:47:19.888399 4892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:19.905124 4892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:19.909030 4892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:19.909253 4892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:19.910077 4892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:19.911330 4892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:19.911401 4892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:19.911489 4892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:19.911531 4892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:19.940048 4892 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:37055
I20260810 06:47:19.940085 5120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:37055 every 8 connection(s)
I20260810 06:47:19.941270 4892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:47:19.949890 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 4892
I20260810 06:47:19.950064 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:47:19.950547 11208 external_mini_cluster.cc:1428] Setting key 0a6f6f9a27c7ab454438230b91b60027
Aug 10 06:47:19 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344439, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:47:19.960829 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:19.943701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:44601 (local address 127.10.242.62:40091)
0810 06:47:19.943862 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:19.943867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:19.945001 (+ 1134us) server_negotiation.cc:407] Connection header received
0810 06:47:19.946825 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:19.946830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:19.946903 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:19.946980 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:19.948381 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:19.949076 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:19.950714 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:19.951291 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:19.951418 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:19.955056 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:19.955062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:19.955066 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:19.955102 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:19.957910 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:19.958625 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:19.958630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:19.958633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:19.958791 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:19.959149 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:19.959154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:19.959157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:19.959483 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:19.959626 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:19.960304 (+ 678us) server_negotiation.cc:299] Negotiation successful
0810 06:47:19.960516 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:47:19.961807 5125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:19.944129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:44601)
0810 06:47:19.944817 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:19.944865 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:19.946557 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:19.947169 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:19.947177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:19.947702 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:19.948233 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:19.948250 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:19.949206 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:19.949209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:19.950552 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:19.950566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:19.951491 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:19.951496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:19.951705 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:19.952326 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:19.952355 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:19.954836 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:19.958032 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:19.958039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:19.958050 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:19.958504 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:19.958887 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:19.958890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:19.958892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:19.959060 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:19.959595 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:19.959601 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:19.959889 (+ 288us) client_negotiation.cc:769] Sending connection context
0810 06:47:19.960178 (+ 289us) client_negotiation.cc:240] Negotiation successful
0810 06:47:19.960578 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":263,"threads_started":1}
I20260810 06:47:19.963225 5121 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40091
I20260810 06:47:19.963541 5121 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:19.964231 5121 heartbeater.cc:511] Master 127.10.242.62:40091 requested a full tablet report, sending...
I20260810 06:47:19.965601 4558 ts_manager.cc:194] Registered new tserver with Master: c6599f4a1d0347a199fa661abf0ac14b (127.10.242.2:37055)
I20260810 06:47:19.966454 4558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:44601
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:47:20.008929 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40091
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:20.333125 5128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:20.333403 5128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:20.333539 5128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:20.340624 5128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:20.340735 5128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:20.340888 5128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:47:20.350248 5128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:20.351452 5128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:20.352677 5128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:20.365485 5149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:20.365619 5142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:20.365628 5135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:20.366442 5128 server_base.cc:1034] running on GCE node
I20260810 06:47:20.367025 5128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:20.367894 5128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:20.369092 5128 hybrid_clock.cc:648] HybridClock initialized: now 1786344440369054 us; error 26 us; skew 500 ppm
I20260810 06:47:20.369395 5128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:20 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344440, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:20.372447 5128 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:47:20.373809 5128 webserver.cc:460] Webserver started at http://127.10.242.3:41427/ using document root <none> and password file <none>
I20260810 06:47:20.374507 5128 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:20.374629 5128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:20.374918 5128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:47:20.377863 5128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "8b96c7d7f6e140d69e17818a8bde2501"
format_stamp: "Formatted at 2026-08-10 06:47:20 on dist-test-slave-zj8q"
server_key: "4bd9e8e851897f7082f8b7a058a3eec1"
server_key_iv: "1ccffc8b415319328e1afcc7b33d4a52"
server_key_version: "encryptionkey@0"
I20260810 06:47:20.378640 5128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "8b96c7d7f6e140d69e17818a8bde2501"
format_stamp: "Formatted at 2026-08-10 06:47:20 on dist-test-slave-zj8q"
server_key: "4bd9e8e851897f7082f8b7a058a3eec1"
server_key_iv: "1ccffc8b415319328e1afcc7b33d4a52"
server_key_version: "encryptionkey@0"
I20260810 06:47:20.383651 5128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 06:47:20.387406 5160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:20.388662 5128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:47:20.388801 5128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "8b96c7d7f6e140d69e17818a8bde2501"
format_stamp: "Formatted at 2026-08-10 06:47:20 on dist-test-slave-zj8q"
server_key: "4bd9e8e851897f7082f8b7a058a3eec1"
server_key_iv: "1ccffc8b415319328e1afcc7b33d4a52"
server_key_version: "encryptionkey@0"
I20260810 06:47:20.388981 5128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:20.408001 5128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:20.412029 5128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:20.412303 5128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:20.413201 5128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:20.414453 5128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:20.414525 5128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:20.414616 5128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:20.414659 5128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:20.444268 5128 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:42007
I20260810 06:47:20.444311 5336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:42007 every 8 connection(s)
I20260810 06:47:20.445426 5128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:47:20.453207 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 5128
I20260810 06:47:20.453404 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:47:20.453830 11208 external_mini_cluster.cc:1428] Setting key 61f3c2c27ba3555aa8d29d8a7289c4eb
Aug 10 06:47:20 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344440, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:47:20.465229 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.447859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:33279 (local address 127.10.242.62:40091)
0810 06:47:20.448032 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.448036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.449166 (+ 1130us) server_negotiation.cc:407] Connection header received
0810 06:47:20.450985 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.450991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.451072 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.451158 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:20.452615 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.453588 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.455460 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.455962 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.456073 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.459501 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:20.459506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:20.459509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:20.459544 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:20.462217 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:20.462983 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:20.462988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:20.462990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:20.463120 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:20.463564 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:20.463568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:20.463570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:20.463841 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:20.463973 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.464654 (+ 681us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.464872 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:47:20.466138 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.448319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:33279)
0810 06:47:20.449003 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:20.449045 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:20.450722 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:20.451406 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:20.451413 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:20.451911 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:20.452442 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.452459 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.453737 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.453740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.455328 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.455341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.456193 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.456201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.456435 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.457083 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:20.457112 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:20.459296 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:20.462342 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:20.462352 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:20.462368 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:20.462857 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:20.463239 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:20.463244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:20.463249 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:20.463431 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:20.463965 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:20.463973 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:20.464298 (+ 325us) client_negotiation.cc:769] Sending connection context
0810 06:47:20.464531 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 06:47:20.465039 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":265,"threads_started":1}
I20260810 06:47:20.467496 5337 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40091
I20260810 06:47:20.467792 5337 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:20.468604 5337 heartbeater.cc:511] Master 127.10.242.62:40091 requested a full tablet report, sending...
I20260810 06:47:20.470067 4882 heartbeater.cc:503] Master 127.10.242.62:40091 was elected leader, sending a full tablet report...
I20260810 06:47:20.470098 4558 ts_manager.cc:194] Registered new tserver with Master: 8b96c7d7f6e140d69e17818a8bde2501 (127.10.242.3:42007)
I20260810 06:47:20.470914 4558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:33279
I20260810 06:47:20.481050 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:47:20.498312 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.485465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36932 (local address 127.10.242.62:40091)
0810 06:47:20.485649 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.485655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.486014 (+ 359us) server_negotiation.cc:407] Connection header received
0810 06:47:20.486221 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.486225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.486286 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.486366 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:20.487934 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.488646 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.490075 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.490592 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.490704 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.493333 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:20.493338 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:20.493341 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:20.493374 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:20.495888 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:20.496552 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:20.496555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:20.496558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:20.496689 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:20.496997 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:20.497000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:20.497002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:20.497323 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:20.497539 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.497869 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.498032 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:47:20.502377 4557 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:47:20.509151 4557 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:47:20.532332 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.521202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37171 (local address 127.0.0.1:43232)
0810 06:47:20.523326 (+ 2124us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:20.523341 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:20.523436 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:20.524635 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:20.524642 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:20.524675 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:20.525084 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.525095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.527339 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.527345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.529939 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.529955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.531315 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.531322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.531507 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.531558 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 06:47:20.531695 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 06:47:20.531888 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2003,"mutex_wait_us":64,"thread_start_us":1167,"threads_started":1}
I20260810 06:47:20.532510 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.520567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42007 (local address 127.0.0.1:42500)
0810 06:47:20.522727 (+ 2160us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:20.522764 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:20.523015 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:20.525065 (+ 2050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:20.525069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:20.525093 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:20.525666 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.525679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.527639 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.527644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.529939 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.529955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.531716 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.531723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.531894 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.531924 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:47:20.532011 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:47:20.532131 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1932,"mutex_wait_us":8,"thread_start_us":2496,"threads_started":1}
I20260810 06:47:20.532817 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.520852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42500 (local address 127.10.242.3:42007)
0810 06:47:20.522101 (+ 1249us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.522111 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.522865 (+ 754us) server_negotiation.cc:407] Connection header received
0810 06:47:20.523300 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.523308 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.523562 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.524755 (+ 1193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:20.525888 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.527464 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.530101 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.531500 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.531627 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.531858 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.532334 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.532502 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1064,"thread_start_us":912,"threads_started":1}
I20260810 06:47:20.534067 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.521877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.10.242.1:37171)
0810 06:47:20.524006 (+ 2129us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.524014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.524041 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:47:20.524147 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.524155 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.524393 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.525204 (+ 811us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:20.525426 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.527027 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.530170 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.531108 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.531237 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.531477 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.532849 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.533821 (+ 972us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1946,"thread_start_us":1806,"threads_started":1}
I20260810 06:47:20.535497 5359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.523733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.10.242.2:37055)
0810 06:47:20.526977 (+ 3244us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.526984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.527517 (+ 533us) server_negotiation.cc:407] Connection header received
0810 06:47:20.527627 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.527634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.527850 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.528030 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:20.528779 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.529748 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.533149 (+ 3401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.534483 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.534620 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.534813 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.535053 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.535206 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3103,"thread_start_us":2979,"threads_started":1}
I20260810 06:47:20.535506 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.524039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:37055 (local address 127.0.0.1:50998)
0810 06:47:20.527246 (+ 3207us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:20.527261 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:20.527381 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:20.528206 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:20.528211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:20.528236 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:20.528625 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.528637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.529890 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.529895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.532969 (+ 3074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.532985 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.534701 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.534708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.534875 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.534906 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:47:20.535006 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:47:20.535195 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3100,"thread_start_us":2969,"threads_started":1}
I20260810 06:47:20.536783 5232 tablet_service.cc:1467] Processing CreateTablet for tablet bd22d28a64e84c9aa04b7c94de54453d (DEFAULT_TABLE table=test-table [id=c15116fc2f08402cb462aacc7c6485a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:20.537365 4765 tablet_service.cc:1467] Processing CreateTablet for tablet bd22d28a64e84c9aa04b7c94de54453d (DEFAULT_TABLE table=test-table [id=c15116fc2f08402cb462aacc7c6485a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:20.538415 5232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd22d28a64e84c9aa04b7c94de54453d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:20.538549 4765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd22d28a64e84c9aa04b7c94de54453d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:20.539234 5004 tablet_service.cc:1467] Processing CreateTablet for tablet bd22d28a64e84c9aa04b7c94de54453d (DEFAULT_TABLE table=test-table [id=c15116fc2f08402cb462aacc7c6485a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:20.540494 5004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd22d28a64e84c9aa04b7c94de54453d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:20.549851 5362 tablet_bootstrap.cc:492] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: Bootstrap starting.
I20260810 06:47:20.552210 5364 tablet_bootstrap.cc:492] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: Bootstrap starting.
I20260810 06:47:20.553608 5362 tablet_bootstrap.cc:654] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:20.554368 5365 tablet_bootstrap.cc:492] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: Bootstrap starting.
I20260810 06:47:20.555130 5362 log.cc:824] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:20.555685 5364 tablet_bootstrap.cc:654] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:20.556936 5364 log.cc:824] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:20.557267 5365 tablet_bootstrap.cc:654] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:20.558434 5365 log.cc:824] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:20.560114 5364 tablet_bootstrap.cc:492] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: No bootstrap required, opened a new log
I20260810 06:47:20.560133 5362 tablet_bootstrap.cc:492] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: No bootstrap required, opened a new log
I20260810 06:47:20.560411 5364 ts_tablet_manager.cc:1397] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 06:47:20.560425 5362 ts_tablet_manager.cc:1397] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260810 06:47:20.561484 5365 tablet_bootstrap.cc:492] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: No bootstrap required, opened a new log
I20260810 06:47:20.561740 5365 ts_tablet_manager.cc:1397] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 06:47:20.568110 5362 raft_consensus.cc:348] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } }
I20260810 06:47:20.568466 5362 raft_consensus.cc:374] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:20.568559 5362 raft_consensus.cc:729] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721bc42effaf4bc3a2a3702d9504b229, State: Initialized, Role: FOLLOWER
I20260810 06:47:20.569137 5362 consensus_queue.cc:260] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } }
I20260810 06:47:20.570894 5362 ts_tablet_manager.cc:1428] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260810 06:47:20.570699 5364 raft_consensus.cc:348] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } }
I20260810 06:47:20.571102 5364 raft_consensus.cc:374] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:20.571193 5364 raft_consensus.cc:729] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b96c7d7f6e140d69e17818a8bde2501, State: Initialized, Role: FOLLOWER
I20260810 06:47:20.571841 5364 consensus_queue.cc:260] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } }
I20260810 06:47:20.571889 5365 raft_consensus.cc:348] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } }
I20260810 06:47:20.572294 5365 raft_consensus.cc:374] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:20.572410 5365 raft_consensus.cc:729] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6599f4a1d0347a199fa661abf0ac14b, State: Initialized, Role: FOLLOWER
I20260810 06:47:20.573031 5337 heartbeater.cc:503] Master 127.10.242.62:40091 was elected leader, sending a full tablet report...
I20260810 06:47:20.573040 5365 consensus_queue.cc:260] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } }
I20260810 06:47:20.573666 5364 ts_tablet_manager.cc:1428] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 06:47:20.574558 5121 heartbeater.cc:503] Master 127.10.242.62:40091 was elected leader, sending a full tablet report...
I20260810 06:47:20.575202 5365 ts_tablet_manager.cc:1428] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260810 06:47:20.633419 5370 raft_consensus.cc:482] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:47:20.633723 5370 raft_consensus.cc:504] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } }
I20260810 06:47:20.635519 5370 leader_election.cc:288] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b96c7d7f6e140d69e17818a8bde2501 (127.10.242.3:42007), c6599f4a1d0347a199fa661abf0ac14b (127.10.242.2:37055)
I20260810 06:47:20.642297 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.635903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42007 (local address 127.10.242.1:51201)
0810 06:47:20.636549 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:20.636565 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:20.636717 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:20.637158 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:20.637161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:20.637186 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:20.637518 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.637531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.638793 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.638798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.640346 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.640358 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.641538 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.641545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.641732 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.641769 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:47:20.641854 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:47:20.641946 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":364,"threads_started":1}
I20260810 06:47:20.642488 5359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.636873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:43917 (local address 127.10.242.2:37055)
0810 06:47:20.637089 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.637094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.637359 (+ 265us) server_negotiation.cc:407] Connection header received
0810 06:47:20.637497 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.637503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.637578 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.637663 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:20.638264 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.639058 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.640765 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.641580 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.641718 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.641785 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.642105 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.642258 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 06:47:20.642597 5379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.636670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:37055 (local address 127.10.242.1:43917)
0810 06:47:20.637265 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:20.637282 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:20.637384 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:20.637692 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:20.637696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:20.637720 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:20.638082 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.638094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.639194 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.639198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.640611 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:20.640625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.641787 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.641791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:20.641935 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.641967 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:47:20.642062 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 06:47:20.642228 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":1,"thread_start_us":270,"threads_started":1}
I20260810 06:47:20.642314 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.635940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:51201 (local address 127.10.242.3:42007)
0810 06:47:20.636169 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.636176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.636663 (+ 487us) server_negotiation.cc:407] Connection header received
0810 06:47:20.636878 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.636884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.636965 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.637053 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:20.637684 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.638654 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.640498 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.641326 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.641446 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.641496 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.641911 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.642037 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:47:20.643505 5041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd22d28a64e84c9aa04b7c94de54453d" candidate_uuid: "721bc42effaf4bc3a2a3702d9504b229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6599f4a1d0347a199fa661abf0ac14b" is_pre_election: true
I20260810 06:47:20.643579 5264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd22d28a64e84c9aa04b7c94de54453d" candidate_uuid: "721bc42effaf4bc3a2a3702d9504b229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b96c7d7f6e140d69e17818a8bde2501" is_pre_election: true
I20260810 06:47:20.643993 5041 raft_consensus.cc:2454] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 721bc42effaf4bc3a2a3702d9504b229 in term 0.
I20260810 06:47:20.644057 5264 raft_consensus.cc:2454] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 721bc42effaf4bc3a2a3702d9504b229 in term 0.
I20260810 06:47:20.644630 4693 leader_election.cc:302] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 721bc42effaf4bc3a2a3702d9504b229, c6599f4a1d0347a199fa661abf0ac14b; no voters:
I20260810 06:47:20.644976 5370 raft_consensus.cc:2781] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:47:20.645113 5370 raft_consensus.cc:482] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:47:20.645229 5370 raft_consensus.cc:3037] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:20.647419 5370 raft_consensus.cc:504] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } }
I20260810 06:47:20.647989 5370 leader_election.cc:288] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [CANDIDATE]: Term 1 election: Requested vote from peers 8b96c7d7f6e140d69e17818a8bde2501 (127.10.242.3:42007), c6599f4a1d0347a199fa661abf0ac14b (127.10.242.2:37055)
I20260810 06:47:20.648382 5264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd22d28a64e84c9aa04b7c94de54453d" candidate_uuid: "721bc42effaf4bc3a2a3702d9504b229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b96c7d7f6e140d69e17818a8bde2501"
I20260810 06:47:20.648478 5041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd22d28a64e84c9aa04b7c94de54453d" candidate_uuid: "721bc42effaf4bc3a2a3702d9504b229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6599f4a1d0347a199fa661abf0ac14b"
I20260810 06:47:20.648558 5264 raft_consensus.cc:3037] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:20.648677 5041 raft_consensus.cc:3037] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:20.650774 5264 raft_consensus.cc:2454] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 721bc42effaf4bc3a2a3702d9504b229 in term 1.
I20260810 06:47:20.650920 5041 raft_consensus.cc:2454] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 721bc42effaf4bc3a2a3702d9504b229 in term 1.
I20260810 06:47:20.651122 4693 leader_election.cc:302] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 721bc42effaf4bc3a2a3702d9504b229, 8b96c7d7f6e140d69e17818a8bde2501; no voters:
I20260810 06:47:20.651372 5370 raft_consensus.cc:2781] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:20.651784 5370 raft_consensus.cc:686] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 1 LEADER]: Becoming Leader. State: Replica: 721bc42effaf4bc3a2a3702d9504b229, State: Running, Role: LEADER
I20260810 06:47:20.652295 5370 consensus_queue.cc:237] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } }
I20260810 06:47:20.656574 4558 catalog_manager.cc:5495] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 reported cstate change: term changed from 0 to 1, leader changed from <none> to 721bc42effaf4bc3a2a3702d9504b229 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "721bc42effaf4bc3a2a3702d9504b229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } health_report { overall_health: HEALTHY } } }
I20260810 06:47:20.689706 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.684522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.10.242.1:37171)
0810 06:47:20.684734 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.684740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.684758 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:47:20.684878 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.684882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.684957 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.685078 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:47:20.685462 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.686209 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.687661 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.688080 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.688186 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.688667 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:47:20.689152 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:47:20.689291 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.689384 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.689462 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 06:47:20.691557 4883 tablet.cc:2369] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:47:20.693061 5124 tablet.cc:2369] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:47:20.696949 5338 tablet.cc:2369] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:47:20.697554 5041 raft_consensus.cc:1261] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [term 1 FOLLOWER]: Refusing update from remote peer 721bc42effaf4bc3a2a3702d9504b229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:20.697748 5264 raft_consensus.cc:1261] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [term 1 FOLLOWER]: Refusing update from remote peer 721bc42effaf4bc3a2a3702d9504b229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:20.698501 5380 consensus_queue.cc:1035] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:20.698843 5370 consensus_queue.cc:1035] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:20.710064 5388 mvcc.cc:205] Tried to move back new op lower bound from 7316866829088190464 to 7316866828915482624. Current Snapshot: MvccSnapshot[applied={T|T < 7316866829088190464}]
I20260810 06:47:20.710698 5386 mvcc.cc:205] Tried to move back new op lower bound from 7316866829088190464 to 7316866828915482624. Current Snapshot: MvccSnapshot[applied={T|T < 7316866829088190464}]
I20260810 06:47:20.713626 5390 mvcc.cc:205] Tried to move back new op lower bound from 7316866829088190464 to 7316866828915482624. Current Snapshot: MvccSnapshot[applied={T|T < 7316866829088190464}]
I20260810 06:47:20.722808 4558 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:20.723090 4558 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:20.726583 4558 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Sending DeleteTablet for 3 replicas of tablet bd22d28a64e84c9aa04b7c94de54453d
I20260810 06:47:20.727670 5232 tablet_service.cc:1514] Processing DeleteTablet for tablet bd22d28a64e84c9aa04b7c94de54453d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:20 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:42500
I20260810 06:47:20.727725 5004 tablet_service.cc:1514] Processing DeleteTablet for tablet bd22d28a64e84c9aa04b7c94de54453d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:20 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:50998
I20260810 06:47:20.727725 4765 tablet_service.cc:1514] Processing DeleteTablet for tablet bd22d28a64e84c9aa04b7c94de54453d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:20 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:43232
I20260810 06:47:20.728654 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 4664
I20260810 06:47:20.743791 5407 tablet_replica.cc:330] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: stopping tablet replica
I20260810 06:47:20.744304 5407 raft_consensus.cc:2229] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:47:20.744998 5407 raft_consensus.cc:2258] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:20.745844 5408 tablet_replica.cc:330] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: stopping tablet replica
I20260810 06:47:20.746286 5408 raft_consensus.cc:2229] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:47:20.746788 5408 raft_consensus.cc:2258] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:20.749020 5406 tablet_replica.cc:330] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: stopping tablet replica
I20260810 06:47:20.749415 5406 raft_consensus.cc:2229] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:47:20.749804 5408 ts_tablet_manager.cc:1905] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:20.749816 5407 ts_tablet_manager.cc:1905] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:20.749890 5406 raft_consensus.cc:2258] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 10 06:47:20 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:47:20.756791 5408 ts_tablet_manager.cc:1918] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:20.756911 5406 ts_tablet_manager.cc:1905] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:20.757086 5408 log.cc:1196] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/bd22d28a64e84c9aa04b7c94de54453d
I20260810 06:47:20.757323 5407 ts_tablet_manager.cc:1918] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:20.757525 5407 log.cc:1196] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/bd22d28a64e84c9aa04b7c94de54453d
I20260810 06:47:20.757853 5408 ts_tablet_manager.cc:1939] T bd22d28a64e84c9aa04b7c94de54453d P c6599f4a1d0347a199fa661abf0ac14b: Deleting consensus metadata
I20260810 06:47:20.758112 5407 ts_tablet_manager.cc:1939] T bd22d28a64e84c9aa04b7c94de54453d P 721bc42effaf4bc3a2a3702d9504b229: Deleting consensus metadata
I20260810 06:47:20.759758 4532 catalog_manager.cc:4841] TS c6599f4a1d0347a199fa661abf0ac14b (127.10.242.2:37055): tablet bd22d28a64e84c9aa04b7c94de54453d (table test-table [id=c15116fc2f08402cb462aacc7c6485a1]) successfully deleted
I20260810 06:47:20.762251 4531 catalog_manager.cc:4841] TS 721bc42effaf4bc3a2a3702d9504b229 (127.10.242.1:37171): tablet bd22d28a64e84c9aa04b7c94de54453d (table test-table [id=c15116fc2f08402cb462aacc7c6485a1]) successfully deleted
I20260810 06:47:20.763489 5406 ts_tablet_manager.cc:1918] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:20.763625 5406 log.cc:1196] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/bd22d28a64e84c9aa04b7c94de54453d
I20260810 06:47:20.764015 5406 ts_tablet_manager.cc:1939] T bd22d28a64e84c9aa04b7c94de54453d P 8b96c7d7f6e140d69e17818a8bde2501: Deleting consensus metadata
I20260810 06:47:20.764997 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.729785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43258 (local address 127.10.242.1:37171)
0810 06:47:20.729984 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.729991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.736741 (+ 6750us) server_negotiation.cc:407] Connection header received
0810 06:47:20.737729 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.737736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.737827 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.737899 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:20.742259 (+ 4360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.743543 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.746889 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.747533 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.747649 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.754790 (+ 7141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:20.754806 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:20.754824 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:20.754882 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:20.759067 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:20.762074 (+ 3007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:20.762083 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:20.762087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:20.762310 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:20.762890 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:20.762895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:20.762899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:20.763344 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:20.763699 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.764391 (+ 692us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.764639 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:47:20.765264 4532 catalog_manager.cc:4841] TS 8b96c7d7f6e140d69e17818a8bde2501 (127.10.242.3:42007): tablet bd22d28a64e84c9aa04b7c94de54453d (table test-table [id=c15116fc2f08402cb462aacc7c6485a1]) successfully deleted
I20260810 06:47:20.766395 4875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:20.877274 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 4664
I20260810 06:47:20.902004 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 4892
Aug 10 06:47:20 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:47:20.916719 5359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:20.902743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51006 (local address 127.10.242.2:37055)
0810 06:47:20.902922 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:20.902928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:20.902970 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:47:20.903199 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:20.903204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:20.903270 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:20.903354 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:20.904749 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.905474 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.906877 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:20.907274 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:20.907381 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:20.911409 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:20.911422 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:20.911437 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:20.911473 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:20.913935 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:20.914697 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:20.914704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:20.914707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:20.914830 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:20.915211 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:20.915214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:20.915216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:20.915593 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:20.915876 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:20.916203 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 06:47:20.916411 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:47:20.917411 5109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:21.004536 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 4892
I20260810 06:47:21.024184 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 5128
Aug 10 06:47:21 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:47:21.039006 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:21.024939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.10.242.3:42007)
0810 06:47:21.025109 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:21.025115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:21.025148 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:47:21.025337 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:21.025341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:21.025409 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:21.025492 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:21.027010 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:21.027736 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:21.029023 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:21.029427 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:21.029539 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:21.033947 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:21.033968 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:21.033991 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:21.034042 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:21.036601 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:21.037259 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:21.037266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:21.037268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:21.037402 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:21.037686 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:21.037689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:21.037691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:21.038019 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:21.038280 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:21.038504 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 06:47:21.038696 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:47:21.039613 5330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:21.126801 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 5128
I20260810 06:47:21.150118 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 4508
I20260810 06:47:21.151393 4608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:21.260584 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 4508
I20260810 06:47:41.288242 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40091
--webserver_interface=127.10.242.62
--webserver_port=40971
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:40091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:47:41.619199 5414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:41.619460 5414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:41.619582 5414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:41.626516 5414 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:47:41.626602 5414 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:47:41.626646 5414 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:47:41.626716 5414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:47:41.626787 5414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:41.626878 5414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:41.626935 5414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:47:41.627009 5414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:47:41.636458 5414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:40091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=40971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:41.637622 5414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:41.638923 5414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:41.662874 5428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:41.662906 5426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:41.662940 5425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:41.663707 5414 server_base.cc:1034] running on GCE node
I20260810 06:47:41.664501 5414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:41.665937 5414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:41.667152 5414 hybrid_clock.cc:648] HybridClock initialized: now 1786344461667067 us; error 58 us; skew 500 ppm
I20260810 06:47:41.667515 5414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:41 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344461, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:41.670931 5414 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:47:41.672525 5414 webserver.cc:460] Webserver started at http://127.10.242.62:40971/ using document root <none> and password file <none>
I20260810 06:47:41.673249 5414 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:41.673373 5414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:41.679091 5414 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:47:41.681890 5439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:41.683215 5414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:47:41.683377 5414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "5eb0b721873049109f85696f031acb11"
format_stamp: "Formatted at 2026-08-10 06:47:18 on dist-test-slave-zj8q"
server_key: "3f8b4040ad05d54d6218f90f6b0c74c3"
server_key_iv: "5b16cb6cddacb4fa7c79cab2ef97fbaf"
server_key_version: "encryptionkey@0"
I20260810 06:47:41.684418 5414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:41.698804 5414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:41.702888 5414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:41.703146 5414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:41.720698 5414 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:40091
I20260810 06:47:41.720745 5527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:40091 every 8 connection(s)
I20260810 06:47:41.722174 5414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:47:41.727010 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 5414
I20260810 06:47:41.728065 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:37171
--local_ip_for_outbound_sockets=127.10.242.1
--tserver_master_addrs=127.10.242.62:40091
--webserver_port=39879
--webserver_interface=127.10.242.1
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:47:41.729878 5528 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:47:41.734234 5528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Bootstrap starting.
I20260810 06:47:41.755234 5528 log.cc:824] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:41.767458 5528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:47:41.767962 5528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Bootstrap complete.
I20260810 06:47:41.776891 5528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:47:41.777704 5528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb0b721873049109f85696f031acb11, State: Initialized, Role: FOLLOWER
I20260810 06:47:41.778362 5528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:47:41.778589 5528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:47:41.778712 5528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:47:41.778872 5528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:47:41.783946 5528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:47:41.784435 5528 leader_election.cc:302] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eb0b721873049109f85696f031acb11; no voters:
I20260810 06:47:41.785068 5528 leader_election.cc:288] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:47:41.785225 5534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:47:41.787292 5528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:47:41.792349 5534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 2 LEADER]: Becoming Leader. State: Replica: 5eb0b721873049109f85696f031acb11, State: Running, Role: LEADER
I20260810 06:47:41.792945 5534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:47:41.795814 5536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eb0b721873049109f85696f031acb11. Latest consensus state: current_term: 2 leader_uuid: "5eb0b721873049109f85696f031acb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } } }
I20260810 06:47:41.795822 5535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5eb0b721873049109f85696f031acb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } } }
I20260810 06:47:41.796247 5536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:41.796324 5535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: This master's current role is: LEADER
I20260810 06:47:41.801999 5549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:47:41.809821 5549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:47:41.811165 5549 catalog_manager.cc:1259] Loaded cluster ID: d91b165ff50b4ac5a67a2e62b17d3901
I20260810 06:47:41.811262 5549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:47:41.815258 5549 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:47:41.816715 5549 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Loaded TSK: 0
I20260810 06:47:41.824434 5560 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:47:41.849678 5549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Generated new TSK 1
I20260810 06:47:41.850098 5549 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:47:42.079401 5530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:42.079684 5530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:42.079804 5530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:42.087205 5530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:42.087316 5530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:42.087476 5530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:47:42.097352 5530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:37171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=39879
--tserver_master_addrs=127.10.242.62:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:42.098687 5530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:42.100001 5530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:42.113412 5572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:42.113444 5574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:42.113838 5580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:42.114104 5530 server_base.cc:1034] running on GCE node
I20260810 06:47:42.114733 5530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:42.115685 5530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:42.116920 5530 hybrid_clock.cc:648] HybridClock initialized: now 1786344462116855 us; error 49 us; skew 500 ppm
I20260810 06:47:42.117231 5530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:42 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:42.120484 5530 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:47:42.122023 5530 webserver.cc:460] Webserver started at http://127.10.242.1:39879/ using document root <none> and password file <none>
I20260810 06:47:42.122745 5530 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:42.122884 5530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:42.128655 5530 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 06:47:42.131597 5595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:42.133008 5530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:47:42.133165 5530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "721bc42effaf4bc3a2a3702d9504b229"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "938f595fb4f9154c6313065b9b035b3e"
server_key_iv: "9e0699b3089ff063ff64e72cb5bfd241"
server_key_version: "encryptionkey@0"
I20260810 06:47:42.134368 5530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:42.152781 5530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:42.156906 5530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:42.157155 5530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:42.158200 5530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:42.159580 5530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:42.159667 5530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:42.159777 5530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:42.159849 5530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:42.191358 5530 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:37171
I20260810 06:47:42.191414 5798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:37171 every 8 connection(s)
I20260810 06:47:42.192775 5530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:47:42.198552 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 5530
I20260810 06:47:42.199332 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:37055
--local_ip_for_outbound_sockets=127.10.242.2
--tserver_master_addrs=127.10.242.62:40091
--webserver_port=43009
--webserver_interface=127.10.242.2
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:47:42 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:47:42.216236 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:42.195872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:35983)
0810 06:47:42.196715 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:42.196760 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:42.198570 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:42.200139 (+ 1569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:42.200146 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:42.200643 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:42.201150 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:42.201167 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.203462 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.203465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:42.204679 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:42.204692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.205967 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.205972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:42.206207 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:42.206809 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:42.206831 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:42.209164 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:42.212389 (+ 3225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:42.212397 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:42.212412 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:42.212870 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:42.213341 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:42.213345 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:42.213347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:42.213522 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:42.214347 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:42.214353 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:42.214653 (+ 300us) client_negotiation.cc:769] Sending connection context
0810 06:47:42.214875 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 06:47:42.215256 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":338,"threads_started":1}
I20260810 06:47:42.217038 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:42.195619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:35983 (local address 127.10.242.62:40091)
0810 06:47:42.196199 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:42.196212 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:42.196891 (+ 679us) server_negotiation.cc:407] Connection header received
0810 06:47:42.199189 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:42.199208 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:42.199654 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:42.200022 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:42.201354 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.203281 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.204820 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.205776 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.205950 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:42.209452 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:42.209459 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:42.209473 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:42.209509 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:42.212241 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:42.213011 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:42.213019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:42.213026 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:42.213239 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:42.213625 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:42.213630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:42.213634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:42.214235 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:42.214411 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:42.215115 (+ 704us) server_negotiation.cc:299] Negotiation successful
0810 06:47:42.215522 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":232,"threads_started":1}
I20260810 06:47:42.219726 5799 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40091
I20260810 06:47:42.220009 5799 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:42.220703 5799 heartbeater.cc:511] Master 127.10.242.62:40091 requested a full tablet report, sending...
I20260810 06:47:42.222836 5470 ts_manager.cc:194] Registered new tserver with Master: 721bc42effaf4bc3a2a3702d9504b229 (127.10.242.1:37171)
I20260810 06:47:42.224375 5470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:35983
W20260810 06:47:42.536887 5803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:42.537146 5803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:42.537278 5803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:42.544270 5803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:42.544384 5803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:42.544567 5803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:47:42.553795 5803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:37055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=43009
--tserver_master_addrs=127.10.242.62:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:42.555025 5803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:42.556299 5803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:42.569000 5819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:42.569029 5816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:42.569345 5815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:42.569634 5803 server_base.cc:1034] running on GCE node
I20260810 06:47:42.570338 5803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:42.571275 5803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:42.572467 5803 hybrid_clock.cc:648] HybridClock initialized: now 1786344462572422 us; error 34 us; skew 500 ppm
I20260810 06:47:42.572762 5803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:42 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:42.575744 5803 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:47:42.577066 5803 webserver.cc:460] Webserver started at http://127.10.242.2:43009/ using document root <none> and password file <none>
I20260810 06:47:42.577725 5803 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:42.577858 5803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:42.583252 5803 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:47:42.585886 5835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:42.587214 5803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:47:42.587399 5803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "c6599f4a1d0347a199fa661abf0ac14b"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "204545b00ded816f6e120921bb9c2a0d"
server_key_iv: "60c886d4448a77f3a3fe9083b69b6cd3"
server_key_version: "encryptionkey@0"
I20260810 06:47:42.588438 5803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:42.602735 5803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:42.606258 5803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:42.606498 5803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:42.607375 5803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:42.608596 5803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:42.608688 5803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:42.608789 5803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:42.608862 5803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:42.639038 5803 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:37055
I20260810 06:47:42.639101 6031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:37055 every 8 connection(s)
I20260810 06:47:42.640437 5803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:47:42.650760 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 5803
I20260810 06:47:42.651490 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:42007
--local_ip_for_outbound_sockets=127.10.242.3
--tserver_master_addrs=127.10.242.62:40091
--webserver_port=41427
--webserver_interface=127.10.242.3
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:47:42 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344462, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:47:42.662544 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:42.642892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:46475 (local address 127.10.242.62:40091)
0810 06:47:42.643119 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:42.643125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:42.644248 (+ 1123us) server_negotiation.cc:407] Connection header received
0810 06:47:42.646014 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:42.646020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:42.646102 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:42.646227 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:42.647671 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.648383 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.649841 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.650349 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.650477 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:42.654741 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:42.654749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:42.654754 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:42.654797 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:42.658073 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:42.659494 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:42.659501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:42.659504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:42.659679 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:42.660314 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:42.660320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:42.660324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:42.660728 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:42.660861 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:42.661898 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0810 06:47:42.662171 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:47:42.664697 6035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:42.643391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:46475)
0810 06:47:42.644095 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:42.644134 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:42.645759 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:42.646411 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:42.646419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:42.646935 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:42.647499 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:42.647524 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.648498 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.648502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:42.649716 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:42.649729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:42.650540 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:42.650545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:42.650767 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:42.651437 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:42.651459 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:42.654479 (+ 3020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:42.658297 (+ 3818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:42.658307 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:42.658323 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:42.659354 (+ 1031us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:42.659843 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:42.659849 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:42.659853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:42.660152 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:42.660983 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:42.660992 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:42.661481 (+ 489us) client_negotiation.cc:769] Sending connection context
0810 06:47:42.662846 (+ 1365us) client_negotiation.cc:240] Negotiation successful
0810 06:47:42.663398 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":287,"threads_started":1}
I20260810 06:47:42.666250 6033 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40091
I20260810 06:47:42.666538 6033 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:42.667263 6033 heartbeater.cc:511] Master 127.10.242.62:40091 requested a full tablet report, sending...
I20260810 06:47:42.668658 5470 ts_manager.cc:194] Registered new tserver with Master: c6599f4a1d0347a199fa661abf0ac14b (127.10.242.2:37055)
I20260810 06:47:42.669435 5470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:46475
W20260810 06:47:42.983300 6037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:47:42.983569 6037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:47:42.983700 6037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:47:42.990724 6037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:47:42.990828 6037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:47:42.990964 6037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:47:43.000173 6037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:42007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=41427
--tserver_master_addrs=127.10.242.62:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:47:43.001386 6037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:47:43.002682 6037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:47:43.015270 6046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:43.015233 6048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:47:43.015304 6051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:47:43.016006 6037 server_base.cc:1034] running on GCE node
I20260810 06:47:43.016571 6037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:47:43.017419 6037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:47:43.018606 6037 hybrid_clock.cc:648] HybridClock initialized: now 1786344463018562 us; error 38 us; skew 500 ppm
I20260810 06:47:43.018889 6037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:47:43 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344463, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:47:43.021821 6037 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:47:43.023123 6037 webserver.cc:460] Webserver started at http://127.10.242.3:41427/ using document root <none> and password file <none>
I20260810 06:47:43.023773 6037 fs_manager.cc:359] Metadata directory not provided
I20260810 06:47:43.023906 6037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:47:43.028893 6037 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:47:43.031471 6064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:43.032660 6037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 06:47:43.032805 6037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "8b96c7d7f6e140d69e17818a8bde2501"
format_stamp: "Formatted at 2026-08-10 06:47:20 on dist-test-slave-zj8q"
server_key: "4bd9e8e851897f7082f8b7a058a3eec1"
server_key_iv: "1ccffc8b415319328e1afcc7b33d4a52"
server_key_version: "encryptionkey@0"
I20260810 06:47:43.033735 6037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:47:43.047973 6037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:47:43.051940 6037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:47:43.052153 6037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:47:43.053007 6037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:47:43.054225 6037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:47:43.054315 6037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:43.054390 6037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:47:43.054446 6037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:47:43.083470 6037 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:42007
I20260810 06:47:43.083536 6260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:42007 every 8 connection(s)
I20260810 06:47:43.084803 6037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:47:43.091285 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 6037
Aug 10 06:47:43 dist-test-slave-zj8q krb5kdc[4491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344463, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:47:43.105844 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.087420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:58241 (local address 127.10.242.62:40091)
0810 06:47:43.087589 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.087593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.088750 (+ 1157us) server_negotiation.cc:407] Connection header received
0810 06:47:43.090511 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.090516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.090582 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.090669 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:43.092158 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.093136 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.095592 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.096025 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.096119 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.099985 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:43.099989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:43.099992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:43.100018 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:43.102756 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.103599 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.103605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.103608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.103753 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.104154 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.104158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.104160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.104520 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:43.104673 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.105336 (+ 663us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.105546 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:47:43.106916 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.087854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:58241)
0810 06:47:43.088593 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.088641 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.090282 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.090857 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.090865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.091391 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:47:43.091959 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.091983 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.093374 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.093380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.095423 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.095441 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.096253 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.096262 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.096487 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.097117 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:47:43.097139 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:47:43.099791 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:47:43.102928 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:43.102937 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:43.102948 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:43.103470 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:43.103857 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:47:43.103860 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:47:43.103862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:47:43.104040 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:47:43.104647 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:47:43.104653 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:47:43.104947 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.105182 (+ 235us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.105553 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":299,"threads_started":1}
I20260810 06:47:43.108258 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.092254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33170 (local address 127.10.242.62:40091)
0810 06:47:43.092735 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.092741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.093965 (+ 1224us) server_negotiation.cc:407] Connection header received
0810 06:47:43.094196 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.094202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.094262 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.094341 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:43.096282 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.097123 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.098970 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.099430 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.099520 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.102566 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:43.102572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:43.102577 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:43.102626 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:43.105492 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.106341 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.106345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.106347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.106454 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.106843 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.106846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.106848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.107112 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:43.107322 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.107725 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.107922 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":226,"threads_started":1}
I20260810 06:47:43.108381 6261 heartbeater.cc:347] Connected to a master server at 127.10.242.62:40091
I20260810 06:47:43.108697 6261 heartbeater.cc:464] Registering TS with master...
I20260810 06:47:43.109485 6261 heartbeater.cc:511] Master 127.10.242.62:40091 requested a full tablet report, sending...
I20260810 06:47:43.110802 5470 ts_manager.cc:194] Registered new tserver with Master: 8b96c7d7f6e140d69e17818a8bde2501 (127.10.242.3:42007)
I20260810 06:47:43.111549 5470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:58241
I20260810 06:47:43.122575 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:47:43.128880 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.123480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33172 (local address 127.10.242.62:40091)
0810 06:47:43.123625 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.123629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.124084 (+ 455us) server_negotiation.cc:407] Connection header received
0810 06:47:43.124287 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.124290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.124410 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.124521 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:47:43.124882 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.125570 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.127134 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.127546 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.127634 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.128115 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:47:43.128406 (+ 291us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 06:47:43.128692 (+ 286us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33172: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 06:47:43.128985 6267 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33172: token expired
W20260810 06:47:43.129522 5345 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.242.62:40091)
I20260810 06:47:43.141603 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.130016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33182 (local address 127.10.242.62:40091)
0810 06:47:43.130174 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.130187 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.130204 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:47:43.130319 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.130322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.130373 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.130442 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:43.131704 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.132380 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.133628 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.133998 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.134081 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.136711 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:43.136717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:43.136722 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:43.136750 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:43.139193 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.139847 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.139851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.139853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.139973 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.140379 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.140383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.140384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.140688 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:43.140870 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.141210 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.141370 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:47:43.144915 5470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:47:43.152163 5470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:47:43.175170 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.163724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37171 (local address 127.0.0.1:60430)
0810 06:47:43.165387 (+ 1663us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.165436 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.165920 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.166639 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.166646 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.166677 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.167068 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.167082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.169321 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.169327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.171988 (+ 2661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.172001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.174044 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.174051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.174317 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.174376 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.174489 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.174683 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1425,"mutex_wait_us":101,"thread_start_us":1946,"threads_started":1}
I20260810 06:47:43.176664 6278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.163840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.10.242.1:37171)
0810 06:47:43.165007 (+ 1167us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.165027 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.165573 (+ 546us) server_negotiation.cc:407] Connection header received
0810 06:47:43.166220 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.166230 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.166423 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.166613 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.167286 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.169155 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.172181 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.173751 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.175868 (+ 2117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.176080 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.176159 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.176326 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":980,"thread_start_us":835,"threads_started":1}
I20260810 06:47:43.176776 6279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.163915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42007 (local address 127.0.0.1:58754)
0810 06:47:43.166531 (+ 2616us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.166547 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.166664 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.167592 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.167597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.167624 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.168229 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.168245 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.170630 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.170636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.173521 (+ 2885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.173538 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.175891 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.175899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.176091 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.176127 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.176207 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.176342 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2499,"mutex_wait_us":3,"thread_start_us":2384,"threads_started":1}
I20260810 06:47:43.177767 6284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.164554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.10.242.3:42007)
0810 06:47:43.165530 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.165540 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.166786 (+ 1246us) server_negotiation.cc:407] Connection header received
0810 06:47:43.166896 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.166903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.167122 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.167308 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.168448 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.170439 (+ 1991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.173718 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.175196 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.175325 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.177148 (+ 1823us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.177228 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.177378 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":634,"threads_started":1}
I20260810 06:47:43.180241 5681 tablet_service.cc:1467] Processing CreateTablet for tablet 302f69ac2e9e489db2f4be3237a5e7e0 (DEFAULT_TABLE table=test-table [id=ea20db1cc1164526ad4b39a2e042770b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:43.181716 5681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302f69ac2e9e489db2f4be3237a5e7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:43.181334 6150 tablet_service.cc:1467] Processing CreateTablet for tablet 302f69ac2e9e489db2f4be3237a5e7e0 (DEFAULT_TABLE table=test-table [id=ea20db1cc1164526ad4b39a2e042770b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:43.182673 6150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302f69ac2e9e489db2f4be3237a5e7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:43.184965 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.166655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:37055 (local address 127.0.0.1:55388)
0810 06:47:43.175607 (+ 8952us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.175624 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.175717 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.178671 (+ 2954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.178676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.178703 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.178856 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.178866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.180645 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.180650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.182429 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.182444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.184269 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.184276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.184441 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.184474 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.184578 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.184714 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8853,"thread_start_us":10700,"threads_started":1}
I20260810 06:47:43.185111 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.167155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.10.242.2:37055)
0810 06:47:43.178071 (+ 10916us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.178080 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.178108 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:47:43.178235 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.178242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.178463 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.178657 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.179004 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.180473 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.182597 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.184088 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.184230 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.184407 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.184627 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.184786 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10745,"thread_start_us":10754,"threads_started":1}
I20260810 06:47:43.188745 5917 tablet_service.cc:1467] Processing CreateTablet for tablet 302f69ac2e9e489db2f4be3237a5e7e0 (DEFAULT_TABLE table=test-table [id=ea20db1cc1164526ad4b39a2e042770b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:47:43.190084 5917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302f69ac2e9e489db2f4be3237a5e7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:47:43.206964 6291 tablet_bootstrap.cc:492] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: Bootstrap starting.
I20260810 06:47:43.207834 6292 tablet_bootstrap.cc:492] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Bootstrap starting.
I20260810 06:47:43.210206 6296 tablet_bootstrap.cc:492] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: Bootstrap starting.
I20260810 06:47:43.211719 6292 tablet_bootstrap.cc:654] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:43.211721 6291 tablet_bootstrap.cc:654] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:43.212989 6296 tablet_bootstrap.cc:654] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: Neither blocks nor log segments found. Creating new log.
I20260810 06:47:43.213003 6292 log.cc:824] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:43.213004 6291 log.cc:824] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:43.214116 6296 log.cc:824] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:47:43.216027 6291 tablet_bootstrap.cc:492] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: No bootstrap required, opened a new log
I20260810 06:47:43.216145 6292 tablet_bootstrap.cc:492] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: No bootstrap required, opened a new log
I20260810 06:47:43.216297 6291 ts_tablet_manager.cc:1397] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260810 06:47:43.216450 6292 ts_tablet_manager.cc:1397] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:47:43.217049 6296 tablet_bootstrap.cc:492] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: No bootstrap required, opened a new log
I20260810 06:47:43.217303 6296 ts_tablet_manager.cc:1397] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 06:47:43.223702 6291 raft_consensus.cc:348] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } }
I20260810 06:47:43.224036 6291 raft_consensus.cc:374] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:43.224148 6291 raft_consensus.cc:729] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b96c7d7f6e140d69e17818a8bde2501, State: Initialized, Role: FOLLOWER
I20260810 06:47:43.224694 6291 consensus_queue.cc:260] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } }
I20260810 06:47:43.225742 6261 heartbeater.cc:503] Master 127.10.242.62:40091 was elected leader, sending a full tablet report...
I20260810 06:47:43.226449 6291 ts_tablet_manager.cc:1428] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
I20260810 06:47:43.226843 6292 raft_consensus.cc:348] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } }
I20260810 06:47:43.227305 6292 raft_consensus.cc:374] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:43.227355 5799 heartbeater.cc:503] Master 127.10.242.62:40091 was elected leader, sending a full tablet report...
I20260810 06:47:43.227442 6292 raft_consensus.cc:729] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721bc42effaf4bc3a2a3702d9504b229, State: Initialized, Role: FOLLOWER
I20260810 06:47:43.227355 6296 raft_consensus.cc:348] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } }
I20260810 06:47:43.227761 6296 raft_consensus.cc:374] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:47:43.227860 6296 raft_consensus.cc:729] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6599f4a1d0347a199fa661abf0ac14b, State: Initialized, Role: FOLLOWER
I20260810 06:47:43.228027 6292 consensus_queue.cc:260] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } }
I20260810 06:47:43.228415 6296 consensus_queue.cc:260] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } }
I20260810 06:47:43.232892 6033 heartbeater.cc:503] Master 127.10.242.62:40091 was elected leader, sending a full tablet report...
I20260810 06:47:43.233183 6292 ts_tablet_manager.cc:1428] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
I20260810 06:47:43.233489 6296 ts_tablet_manager.cc:1428] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
W20260810 06:47:43.336376 6262 tablet.cc:2369] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:47:43.365406 6302 raft_consensus.cc:482] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:47:43.365747 6302 raft_consensus.cc:504] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } }
I20260810 06:47:43.367409 6302 leader_election.cc:288] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b96c7d7f6e140d69e17818a8bde2501 (127.10.242.3:42007), c6599f4a1d0347a199fa661abf0ac14b (127.10.242.2:37055)
I20260810 06:47:43.375547 6284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.367838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:35383 (local address 127.10.242.3:42007)
0810 06:47:43.368014 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.368019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.368545 (+ 526us) server_negotiation.cc:407] Connection header received
0810 06:47:43.368776 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.368781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.368862 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.368958 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.369701 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.370765 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.373262 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.374509 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.374670 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.374735 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.375167 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.375293 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:47:43.375547 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.368877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:53835 (local address 127.10.242.2:37055)
0810 06:47:43.369079 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.369084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.369274 (+ 190us) server_negotiation.cc:407] Connection header received
0810 06:47:43.369461 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.369465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.369521 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.369590 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:47:43.370065 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.371031 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.373262 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.374501 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.374662 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.374735 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.375167 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.375293 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:47:43.375752 6306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.368589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:37055 (local address 127.10.242.1:53835)
0810 06:47:43.369162 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.369202 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.369353 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.369627 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.369632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.369655 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.369925 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.369933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.371140 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.371144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.373148 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.373158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.374767 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.374772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.374977 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.375043 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.375139 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.375286 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":53,"spinlock_wait_cycles":6656,"thread_start_us":293,"threads_started":1}
I20260810 06:47:43.375932 6304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.367774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42007 (local address 127.10.242.1:35383)
0810 06:47:43.368456 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.368475 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:47:43.368614 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:47:43.369111 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:47:43.369116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:47:43.369143 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:47:43.369549 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.369559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.370897 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.370900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.373102 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:47:43.373119 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.374742 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.374752 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:47:43.374986 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.375045 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 06:47:43.375139 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 06:47:43.375301 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":78,"spinlock_wait_cycles":7040,"thread_start_us":349,"threads_started":1}
I20260810 06:47:43.376801 5955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302f69ac2e9e489db2f4be3237a5e7e0" candidate_uuid: "721bc42effaf4bc3a2a3702d9504b229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6599f4a1d0347a199fa661abf0ac14b" is_pre_election: true
I20260810 06:47:43.376796 6183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302f69ac2e9e489db2f4be3237a5e7e0" candidate_uuid: "721bc42effaf4bc3a2a3702d9504b229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b96c7d7f6e140d69e17818a8bde2501" is_pre_election: true
I20260810 06:47:43.377259 6183 raft_consensus.cc:2454] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 721bc42effaf4bc3a2a3702d9504b229 in term 0.
I20260810 06:47:43.377249 5955 raft_consensus.cc:2454] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 721bc42effaf4bc3a2a3702d9504b229 in term 0.
I20260810 06:47:43.377892 5597 leader_election.cc:302] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 721bc42effaf4bc3a2a3702d9504b229, c6599f4a1d0347a199fa661abf0ac14b; no voters:
I20260810 06:47:43.378281 6302 raft_consensus.cc:2781] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:47:43.378436 6302 raft_consensus.cc:482] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:47:43.378549 6302 raft_consensus.cc:3037] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:43.380693 6302 raft_consensus.cc:504] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } }
I20260810 06:47:43.381258 6302 leader_election.cc:288] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [CANDIDATE]: Term 1 election: Requested vote from peers 8b96c7d7f6e140d69e17818a8bde2501 (127.10.242.3:42007), c6599f4a1d0347a199fa661abf0ac14b (127.10.242.2:37055)
I20260810 06:47:43.381682 6183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302f69ac2e9e489db2f4be3237a5e7e0" candidate_uuid: "721bc42effaf4bc3a2a3702d9504b229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b96c7d7f6e140d69e17818a8bde2501"
I20260810 06:47:43.381734 5955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302f69ac2e9e489db2f4be3237a5e7e0" candidate_uuid: "721bc42effaf4bc3a2a3702d9504b229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6599f4a1d0347a199fa661abf0ac14b"
I20260810 06:47:43.381901 5955 raft_consensus.cc:3037] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:43.381902 6183 raft_consensus.cc:3037] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:47:43.385013 5955 raft_consensus.cc:2454] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 721bc42effaf4bc3a2a3702d9504b229 in term 1.
I20260810 06:47:43.385012 6183 raft_consensus.cc:2454] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 721bc42effaf4bc3a2a3702d9504b229 in term 1.
I20260810 06:47:43.385381 5597 leader_election.cc:302] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 721bc42effaf4bc3a2a3702d9504b229, c6599f4a1d0347a199fa661abf0ac14b; no voters:
I20260810 06:47:43.385664 6302 raft_consensus.cc:2781] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:47:43.386070 6302 raft_consensus.cc:686] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 1 LEADER]: Becoming Leader. State: Replica: 721bc42effaf4bc3a2a3702d9504b229, State: Running, Role: LEADER
I20260810 06:47:43.386566 6302 consensus_queue.cc:237] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } }
I20260810 06:47:43.390936 5467 catalog_manager.cc:5495] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 reported cstate change: term changed from 0 to 1, leader changed from <none> to 721bc42effaf4bc3a2a3702d9504b229 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "721bc42effaf4bc3a2a3702d9504b229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "721bc42effaf4bc3a2a3702d9504b229" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 } health_report { overall_health: UNKNOWN } } }
W20260810 06:47:43.392197 6034 tablet.cc:2369] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:47:43.417593 6278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.412260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60440 (local address 127.10.242.1:37171)
0810 06:47:43.412447 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.412453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.412539 (+ 86us) server_negotiation.cc:407] Connection header received
0810 06:47:43.412784 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.412788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.412871 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.412995 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:47:43.413373 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.414083 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.415518 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.415927 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.416032 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.416545 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:47:43.417029 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:47:43.417167 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.417265 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.417344 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:47:43.425249 5955 raft_consensus.cc:1261] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [term 1 FOLLOWER]: Refusing update from remote peer 721bc42effaf4bc3a2a3702d9504b229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:43.425273 6183 raft_consensus.cc:1261] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [term 1 FOLLOWER]: Refusing update from remote peer 721bc42effaf4bc3a2a3702d9504b229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:47:43.426131 6307 consensus_queue.cc:1035] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b96c7d7f6e140d69e17818a8bde2501" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:43.426502 6302 consensus_queue.cc:1035] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6599f4a1d0347a199fa661abf0ac14b" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:47:43.437533 6311 mvcc.cc:205] Tried to move back new op lower bound from 7316866922180419584 to 7316866922034929664. Current Snapshot: MvccSnapshot[applied={T|T < 7316866922180419584}]
I20260810 06:47:43.437029 6309 mvcc.cc:205] Tried to move back new op lower bound from 7316866922180419584 to 7316866922034929664. Current Snapshot: MvccSnapshot[applied={T|T < 7316866922180419584}]
I20260810 06:47:43.451557 5467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:43.451807 5467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:47:43.455113 5467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Sending DeleteTablet for 3 replicas of tablet 302f69ac2e9e489db2f4be3237a5e7e0
I20260810 06:47:43.456204 5681 tablet_service.cc:1514] Processing DeleteTablet for tablet 302f69ac2e9e489db2f4be3237a5e7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:43 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:60430
I20260810 06:47:43.456249 5917 tablet_service.cc:1514] Processing DeleteTablet for tablet 302f69ac2e9e489db2f4be3237a5e7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:43 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:55388
I20260810 06:47:43.456418 6150 tablet_service.cc:1514] Processing DeleteTablet for tablet 302f69ac2e9e489db2f4be3237a5e7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:47:43 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:58754
I20260810 06:47:43.457515 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 5530
I20260810 06:47:43.466807 6333 tablet_replica.cc:330] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: stopping tablet replica
I20260810 06:47:43.467365 6333 raft_consensus.cc:2229] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:47:43.467914 6333 raft_consensus.cc:2258] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:43.468750 6330 tablet_replica.cc:330] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: stopping tablet replica
I20260810 06:47:43.469272 6331 tablet_replica.cc:330] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: stopping tablet replica
I20260810 06:47:43.469751 6331 raft_consensus.cc:2229] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:47:43.470253 6330 raft_consensus.cc:2229] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:47:43.470357 6333 ts_tablet_manager.cc:1905] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:43.471029 6331 raft_consensus.cc:2258] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:43.471464 6330 raft_consensus.cc:2258] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:47:43.473809 6330 ts_tablet_manager.cc:1905] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:43.475956 6331 ts_tablet_manager.cc:1905] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:47:43.477923 6333 ts_tablet_manager.cc:1918] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:43.478130 6333 log.cc:1196] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/302f69ac2e9e489db2f4be3237a5e7e0
I20260810 06:47:43.478801 6333 ts_tablet_manager.cc:1939] T 302f69ac2e9e489db2f4be3237a5e7e0 P 8b96c7d7f6e140d69e17818a8bde2501: Deleting consensus metadata
I20260810 06:47:43.479303 6278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.459332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.10.242.1:37171)
0810 06:47:43.459498 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.459505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.459523 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:47:43.459578 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.459584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.459667 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.459759 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:43.461891 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.463341 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.465467 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.466140 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.466354 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.471227 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:43.471241 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:43.471256 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:43.471290 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:43.475027 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.475923 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.475932 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.475935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.476087 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.476545 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.476550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.476553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.477006 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:43.477360 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.478594 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.478920 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:47:43.479946 5786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:43.481011 5442 catalog_manager.cc:4841] TS 8b96c7d7f6e140d69e17818a8bde2501 (127.10.242.3:42007): tablet 302f69ac2e9e489db2f4be3237a5e7e0 (table test-table [id=ea20db1cc1164526ad4b39a2e042770b]) successfully deleted
I20260810 06:47:43.487126 6330 ts_tablet_manager.cc:1918] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:47:43.487285 6330 log.cc:1196] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/302f69ac2e9e489db2f4be3237a5e7e0
I20260810 06:47:43.487834 6330 ts_tablet_manager.cc:1939] T 302f69ac2e9e489db2f4be3237a5e7e0 P c6599f4a1d0347a199fa661abf0ac14b: Deleting consensus metadata
I20260810 06:47:43.489490 5442 catalog_manager.cc:4841] TS c6599f4a1d0347a199fa661abf0ac14b (127.10.242.2:37055): tablet 302f69ac2e9e489db2f4be3237a5e7e0 (table test-table [id=ea20db1cc1164526ad4b39a2e042770b]) successfully deleted
I20260810 06:47:43.577086 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 5530
W20260810 06:47:43.603262 5440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.1:37171 (error 108)
I20260810 06:47:43.603490 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 5803
I20260810 06:47:43.605098 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.604688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37171 (local address 127.0.0.1:60466)
0810 06:47:43.604846 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.604943 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:47:43.605337 5440 catalog_manager.cc:4584] TS 721bc42effaf4bc3a2a3702d9504b229 (127.10.242.1:37171): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 302f69ac2e9e489db2f4be3237a5e7e0: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37171: connect: Connection refused (error 111)
I20260810 06:47:43.617070 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.604305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55390 (local address 127.10.242.2:37055)
0810 06:47:43.604486 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.604492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.604607 (+ 115us) server_negotiation.cc:407] Connection header received
0810 06:47:43.604811 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.604817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.604889 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.604993 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:43.606628 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.607356 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.608667 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.609064 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.609173 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.611844 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:43.611858 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:43.611873 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:43.611909 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:43.614446 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.615106 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.615114 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.615117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.615240 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.615592 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.615597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.615601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.615972 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:43.616261 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.616556 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.616772 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 06:47:43.617666 6021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:43.658911 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.658456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37171 (local address 127.0.0.1:60482)
0810 06:47:43.658668 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.658762 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 06:47:43.659875 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.659549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37171 (local address 127.0.0.1:60490)
0810 06:47:43.659678 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.659745 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:47:43.705833 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 5803
I20260810 06:47:43.720436 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.719933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37171 (local address 127.0.0.1:60498)
0810 06:47:43.720147 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.720265 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:47:43.721576 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.721155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37171 (local address 127.0.0.1:60508)
0810 06:47:43.721320 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.721408 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:47:43.733744 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 6037
I20260810 06:47:43.747193 6284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.734503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.10.242.3:42007)
0810 06:47:43.734679 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:47:43.734684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:47:43.734724 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:47:43.734934 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:47:43.734938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:47:43.735006 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:47:43.735089 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:47:43.736507 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.737286 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.738828 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:47:43.739220 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:47:43.739330 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:47:43.741809 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:47:43.741825 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:47:43.741844 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:47:43.741881 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:47:43.744337 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.745001 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.745010 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.745012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.745179 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:47:43.745556 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:47:43.745561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:47:43.745563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:47:43.745913 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:47:43.746259 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 06:47:43.746676 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 06:47:43.746881 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:47:43.747776 6254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:43.802404 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.801783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37171 (local address 127.0.0.1:60512)
0810 06:47:43.802016 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.802198 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 06:47:43.803521 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:47:43.803076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37171 (local address 127.0.0.1:60514)
0810 06:47:43.803280 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 06:47:43.803367 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 06:47:43.835897 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 6037
I20260810 06:47:43.858680 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 5414
I20260810 06:47:43.859838 5520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:47:43.974344 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 5414
I20260810 06:48:04.004302 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40091
--webserver_interface=127.10.242.62
--webserver_port=40971
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:40091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:04.354139 6345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:04.354446 6345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:04.354588 6345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:04.361686 6345 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:48:04.361776 6345 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:48:04.361837 6345 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:48:04.361907 6345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:04.361982 6345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:04.362077 6345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:04.362141 6345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:04.362232 6345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:04.371933 6345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:40091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:40091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=40971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:04.373183 6345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:04.374585 6345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:04.386101 6356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:04.386173 6362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:04.386183 6358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:04.387293 6345 server_base.cc:1034] running on GCE node
I20260810 06:48:04.388057 6345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:04.389385 6345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:04.390627 6345 hybrid_clock.cc:648] HybridClock initialized: now 1786344484390568 us; error 28 us; skew 500 ppm
I20260810 06:48:04.390934 6345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:04 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344484, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:04.397009 6345 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:48:04.398488 6345 webserver.cc:460] Webserver started at http://127.10.242.62:40971/ using document root <none> and password file <none>
I20260810 06:48:04.399170 6345 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:04.399277 6345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:04.404601 6345 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:48:04.407380 6373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:04.408620 6345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:48:04.408784 6345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "5eb0b721873049109f85696f031acb11"
format_stamp: "Formatted at 2026-08-10 06:47:18 on dist-test-slave-zj8q"
server_key: "3f8b4040ad05d54d6218f90f6b0c74c3"
server_key_iv: "5b16cb6cddacb4fa7c79cab2ef97fbaf"
server_key_version: "encryptionkey@0"
I20260810 06:48:04.409725 6345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:04.437184 6345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:04.441022 6345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:04.441280 6345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:04.458958 6345 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:40091
I20260810 06:48:04.458988 6464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:40091 every 8 connection(s)
I20260810 06:48:04.460345 6345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:04.466975 6466 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:48:04.468087 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 6345
I20260810 06:48:04.468918 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:37171
--local_ip_for_outbound_sockets=127.10.242.1
--tserver_master_addrs=127.10.242.62:40091
--webserver_port=39879
--webserver_interface=127.10.242.1
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:48:04.470170 6466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Bootstrap starting.
I20260810 06:48:04.490206 6466 log.cc:824] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:04.505995 6466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:48:04.506523 6466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Bootstrap complete.
I20260810 06:48:04.515712 6466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:48:04.516569 6466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eb0b721873049109f85696f031acb11, State: Initialized, Role: FOLLOWER
I20260810 06:48:04.517115 6466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:48:04.517341 6466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:04.517448 6466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:04.517614 6466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:48:04.521303 6466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:48:04.521793 6466 leader_election.cc:302] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eb0b721873049109f85696f031acb11; no voters:
I20260810 06:48:04.522418 6466 leader_election.cc:288] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:48:04.522570 6475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:48:04.524505 6466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:04.529218 6475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [term 3 LEADER]: Becoming Leader. State: Replica: 5eb0b721873049109f85696f031acb11, State: Running, Role: LEADER
I20260810 06:48:04.529794 6475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } }
I20260810 06:48:04.532523 6476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5eb0b721873049109f85696f031acb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } } }
I20260810 06:48:04.532886 6476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:04.533425 6477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eb0b721873049109f85696f031acb11. Latest consensus state: current_term: 3 leader_uuid: "5eb0b721873049109f85696f031acb11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eb0b721873049109f85696f031acb11" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 40091 } } }
I20260810 06:48:04.533732 6477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:04.543648 6492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:04.555718 6492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:04.557238 6492 catalog_manager.cc:1259] Loaded cluster ID: d91b165ff50b4ac5a67a2e62b17d3901
I20260810 06:48:04.557394 6492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:04.561734 6492 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:04.563426 6492 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Loaded TSK: 0
I20260810 06:48:04.563659 6492 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5eb0b721873049109f85696f031acb11: Loaded TSK: 1
F20260810 06:48:04.564854 6492 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2da9bbf9ad google::LogMessage::Fail() at ??:0
@ 0x7f2da9bc288a google::LogMessage::SendToLog() at ??:0
@ 0x7f2da9bbf4de google::LogMessage::Flush() at ??:0
@ 0x7f2da9bc0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2dad9237d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2dad91273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f2dbf5ee1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f2dbf5ed783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f2dbf6792a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f2dbf6790c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f2dbf6c35e9 std::function<>::operator()() at ??:0
@ 0x7f2dbf5f1cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f2dbf5efc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2dbf677610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f2dbf677439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2dac80b345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2dac80e91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f2dac80e799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2dac7e9eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2dafac36db start_thread at ??:0
@ 0x7f2da8a5961f clone at ??:0
W20260810 06:48:04.868158 6468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:04.868432 6468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:04.868562 6468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:04.875633 6468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:04.875739 6468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:04.875900 6468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:48:04.885738 6468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:37171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=39879
--tserver_master_addrs=127.10.242.62:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:04.887017 6468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:04.888298 6468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:04.900846 6554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:04.900884 6556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:04.900914 6558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:04.901741 6468 server_base.cc:1034] running on GCE node
I20260810 06:48:04.902336 6468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:04.903179 6468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:04.904390 6468 hybrid_clock.cc:648] HybridClock initialized: now 1786344484904340 us; error 40 us; skew 500 ppm
I20260810 06:48:04.904721 6468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:04 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344484, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:04.907749 6468 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:48:04.909111 6468 webserver.cc:460] Webserver started at http://127.10.242.1:39879/ using document root <none> and password file <none>
I20260810 06:48:04.909757 6468 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:04.909879 6468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:04.915045 6468 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:48:04.918040 6574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:04.919298 6468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:48:04.919453 6468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "721bc42effaf4bc3a2a3702d9504b229"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "938f595fb4f9154c6313065b9b035b3e"
server_key_iv: "9e0699b3089ff063ff64e72cb5bfd241"
server_key_version: "encryptionkey@0"
I20260810 06:48:04.920373 6468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:04.939733 6468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:04.943763 6468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:04.944031 6468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:04.945041 6468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:04.946893 6583 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260810 06:48:04.952150 6583 ts_tablet_manager.cc:1852] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260810 06:48:04.953132 6583 ts_tablet_manager.cc:1905] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:04.960556 6583 ts_tablet_manager.cc:1918] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:04.960750 6583 log.cc:1196] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/302f69ac2e9e489db2f4be3237a5e7e0
I20260810 06:48:04.961269 6583 ts_tablet_manager.cc:1939] T 302f69ac2e9e489db2f4be3237a5e7e0 P 721bc42effaf4bc3a2a3702d9504b229: Deleting consensus metadata
I20260810 06:48:04.961649 6468 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260810 06:48:04.961781 6468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260810 06:48:04.961910 6468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:04.961985 6468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:04.992270 6468 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:37171
I20260810 06:48:04.992324 6755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:37171 every 8 connection(s)
I20260810 06:48:04.993532 6468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:04.998351 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:04.996479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:33577)
0810 06:48:04.997213 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:04.997439 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":266,"threads_started":1}
I20260810 06:48:04.998562 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 6468
W20260810 06:48:04.998927 6757 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:40091 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.62:40091: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
I20260810 06:48:04.999341 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:37055
--local_ip_for_outbound_sockets=127.10.242.2
--tserver_master_addrs=127.10.242.62:40091
--webserver_port=43009
--webserver_interface=127.10.242.2
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:48:04.999776 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:04.999426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:47013)
0810 06:48:04.999585 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:04.999664 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:48:05.000576 6760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.000269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:58739)
0810 06:48:05.000391 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.000465 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:48:05.000782 6757 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:48:05.332912 6761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:05.333184 6761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:05.333324 6761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:05.340443 6761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:05.340544 6761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:05.340764 6761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:48:05.350319 6761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:37055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=43009
--tserver_master_addrs=127.10.242.62:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:05.351526 6761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:05.352839 6761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:05.365381 6776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:05.365443 6777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:05.365473 6780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:05.366467 6761 server_base.cc:1034] running on GCE node
I20260810 06:48:05.367063 6761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:05.367959 6761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:05.369159 6761 hybrid_clock.cc:648] HybridClock initialized: now 1786344485369110 us; error 36 us; skew 500 ppm
I20260810 06:48:05.369448 6761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:05 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344485, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:05.372560 6761 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:48:05.373881 6761 webserver.cc:460] Webserver started at http://127.10.242.2:43009/ using document root <none> and password file <none>
I20260810 06:48:05.374559 6761 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:05.374663 6761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:05.379760 6761 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:48:05.382414 6796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:05.383750 6761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:48:05.383891 6761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "c6599f4a1d0347a199fa661abf0ac14b"
format_stamp: "Formatted at 2026-08-10 06:47:19 on dist-test-slave-zj8q"
server_key: "204545b00ded816f6e120921bb9c2a0d"
server_key_iv: "60c886d4448a77f3a3fe9083b69b6cd3"
server_key_version: "encryptionkey@0"
I20260810 06:48:05.384814 6761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:05.407224 6761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:05.411231 6761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:05.411466 6761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:05.412369 6761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:05.413627 6761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:05.413724 6761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:05.413825 6761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:05.413892 6761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:05.445102 6761 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:37055
I20260810 06:48:05.445174 6988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:37055 every 8 connection(s)
I20260810 06:48:05.446573 6761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:48:05.447502 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 6761
I20260810 06:48:05.448268 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:42007
--local_ip_for_outbound_sockets=127.10.242.3
--tserver_master_addrs=127.10.242.62:40091
--webserver_port=41427
--webserver_interface=127.10.242.3
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:45647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:48:05.451946 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.449730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:50519)
0810 06:48:05.450572 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.450855 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":333,"threads_started":1}
W20260810 06:48:05.452639 6990 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:40091 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.62:40091: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
I20260810 06:48:05.453445 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.453105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:49033)
0810 06:48:05.453249 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.453326 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:48:05.454314 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.453947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:35695)
0810 06:48:05.454080 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.454194 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:48:05.454581 6990 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:48:05.774143 6994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:05.774413 6994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:05.774545 6994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:05.781491 6994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:05.781596 6994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:05.781746 6994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:48:05.791515 6994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:45647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:42007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=41427
--tserver_master_addrs=127.10.242.62:40091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:05.792717 6994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:05.794005 6994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:05.806520 7002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:05.806497 7006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:05.806452 7004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:05.807477 6994 server_base.cc:1034] running on GCE node
I20260810 06:48:05.808043 6994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:05.808928 6994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:05.810132 6994 hybrid_clock.cc:648] HybridClock initialized: now 1786344485810082 us; error 40 us; skew 500 ppm
I20260810 06:48:05.810456 6994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:05 dist-test-slave-zj8q krb5kdc[4491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344485, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:05.813505 6994 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:48:05.814857 6994 webserver.cc:460] Webserver started at http://127.10.242.3:41427/ using document root <none> and password file <none>
I20260810 06:48:05.815478 6994 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:05.815578 6994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:05.820676 6994 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:48:05.823242 7014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:05.824678 6994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:48:05.824836 6994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "8b96c7d7f6e140d69e17818a8bde2501"
format_stamp: "Formatted at 2026-08-10 06:47:20 on dist-test-slave-zj8q"
server_key: "4bd9e8e851897f7082f8b7a058a3eec1"
server_key_iv: "1ccffc8b415319328e1afcc7b33d4a52"
server_key_version: "encryptionkey@0"
I20260810 06:48:05.825829 6994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:05.844501 6994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:05.848498 6994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:05.848747 6994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:05.849725 6994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:05.851042 6994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:05.851138 6994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:05.851256 6994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:05.851338 6994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:05.881240 6994 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:42007
I20260810 06:48:05.881337 7198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:42007 every 8 connection(s)
I20260810 06:48:05.882697 6994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:48:05.885582 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 6994
I20260810 06:48:05.887698 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.885543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:58519)
0810 06:48:05.886359 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.886594 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":321,"threads_started":1}
W20260810 06:48:05.888300 7199 heartbeater.cc:644] Failed to heartbeat to 127.10.242.62:40091 (0 consecutive failures): Network error: Failed to ping master at 127.10.242.62:40091: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
I20260810 06:48:05.889081 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.888712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:45791)
0810 06:48:05.888888 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.888958 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 06:48:05.889823 7201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:05.889510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:33847)
0810 06:48:05.889652 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:05.889723 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:48:05.890043 7199 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:48:06.002061 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:06.001296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:39479)
0810 06:48:06.001741 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:06.001828 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":211,"threads_started":1}
I20260810 06:48:06.455762 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:06.455009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:43859)
0810 06:48:06.455448 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:06.455533 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":221,"threads_started":1}
I20260810 06:48:06.891269 7206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:06.890529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:58501)
0810 06:48:06.890948 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:06.891043 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":200,"threads_started":1}
I20260810 06:48:07.003661 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:07.002971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:34877)
0810 06:48:07.003414 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:07.003494 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":235,"threads_started":1}
I20260810 06:48:07.457270 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:07.456543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:52895)
0810 06:48:07.456987 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:07.457066 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":221,"threads_started":1}
I20260810 06:48:07.892920 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:07.892166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:37825)
0810 06:48:07.892648 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:07.892746 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":226,"threads_started":1}
I20260810 06:48:08.005419 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:08.004593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:47607)
0810 06:48:08.005142 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:08.005241 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":276,"threads_started":1}
I20260810 06:48:08.458891 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:08.458189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:34995)
0810 06:48:08.458621 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:08.458708 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":220,"threads_started":1}
I20260810 06:48:08.894413 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:08.893673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:57773)
0810 06:48:08.894126 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:08.894246 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":243,"threads_started":1}
I20260810 06:48:09.007047 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.006316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:33311)
0810 06:48:09.006806 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.006886 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":265,"threads_started":1}
I20260810 06:48:09.460538 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.459666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:41015)
0810 06:48:09.460202 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.460287 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":271,"threads_started":1}
I20260810 06:48:09.896023 7219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:09.895217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:52029)
0810 06:48:09.895697 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:09.895804 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":209,"threads_started":1}
I20260810 06:48:10.008605 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:10.007873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:39625)
0810 06:48:10.008313 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:10.008390 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":216,"threads_started":1}
I20260810 06:48:10.462258 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:10.461450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:51997)
0810 06:48:10.461903 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:10.462003 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":232,"threads_started":1}
I20260810 06:48:10.897639 7222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:10.896891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:57833)
0810 06:48:10.897389 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:10.897472 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":252,"threads_started":1}
I20260810 06:48:11.010139 7226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:11.009458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:48465)
0810 06:48:11.009896 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:11.009977 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":227,"threads_started":1}
I20260810 06:48:11.463760 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:11.463072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:52671)
0810 06:48:11.463484 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:11.463572 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":208,"threads_started":1}
I20260810 06:48:11.899096 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:11.898388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:38521)
0810 06:48:11.898856 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:11.898945 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":244,"threads_started":1}
I20260810 06:48:12.011781 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:12.010986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:60025)
0810 06:48:12.011494 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:12.011572 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":239,"threads_started":1}
I20260810 06:48:12.465453 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:12.464626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:40385)
0810 06:48:12.465143 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:12.465228 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":222,"threads_started":1}
I20260810 06:48:12.900624 7232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:12.899894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:42945)
0810 06:48:12.900339 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:12.900415 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":193,"threads_started":1}
I20260810 06:48:13.013383 7233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:13.012673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:42583)
0810 06:48:13.013120 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:13.013212 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":226,"threads_started":1}
I20260810 06:48:13.467038 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:13.466323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:50797)
0810 06:48:13.466766 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:13.466871 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":227,"threads_started":1}
I20260810 06:48:13.902114 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:13.901414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:39637)
0810 06:48:13.901875 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:13.901953 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":247,"threads_started":1}
I20260810 06:48:14.015201 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:14.014243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:39743)
0810 06:48:14.014833 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:14.014967 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":273,"threads_started":1}
I20260810 06:48:14.468541 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:14.467851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:44369)
0810 06:48:14.468295 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:14.468373 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":202,"threads_started":1}
I20260810 06:48:14.903652 7241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:14.902935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:40887)
0810 06:48:14.903395 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:14.903474 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":232,"threads_started":1}
I20260810 06:48:15.016842 7242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:15.016098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:56813)
0810 06:48:15.016578 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:15.016658 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":264,"threads_started":1}
I20260810 06:48:15.470204 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:15.469416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:47181)
0810 06:48:15.469922 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:15.470002 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":298,"threads_started":1}
I20260810 06:48:15.905174 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:15.904498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:52267)
0810 06:48:15.904921 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:15.904999 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":229,"threads_started":1}
I20260810 06:48:16.018778 7246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:16.017758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:44803)
0810 06:48:16.018425 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:16.018539 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":403,"threads_started":1}
I20260810 06:48:16.471772 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:16.471052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:58705)
0810 06:48:16.471478 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:16.471573 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":218,"threads_started":1}
I20260810 06:48:16.906816 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:16.906023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:43269)
0810 06:48:16.906536 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:16.906631 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":316,"threads_started":1}
I20260810 06:48:17.020452 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:17.019738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:45271)
0810 06:48:17.020196 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:17.020291 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":234,"threads_started":1}
I20260810 06:48:17.473294 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:17.472638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:57955)
0810 06:48:17.473057 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:17.473137 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":211,"threads_started":1}
I20260810 06:48:17.908303 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:17.907651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:35287)
0810 06:48:17.908068 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:17.908146 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":215,"threads_started":1}
I20260810 06:48:18.022056 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:18.021298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:42817)
0810 06:48:18.021766 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:18.021867 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":216,"threads_started":1}
I20260810 06:48:18.474861 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:18.474094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:41217)
0810 06:48:18.474578 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:18.474674 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":256,"threads_started":1}
I20260810 06:48:18.909868 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:18.909132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:33831)
0810 06:48:18.909568 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:18.909679 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":213,"threads_started":1}
I20260810 06:48:19.023571 7265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:19.022892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:35949)
0810 06:48:19.023313 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:19.023394 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":209,"threads_started":1}
I20260810 06:48:19.476317 7266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:19.475632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:33301)
0810 06:48:19.476056 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:19.476136 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":223,"threads_started":1}
I20260810 06:48:19.911381 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:19.910684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:44915)
0810 06:48:19.911117 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:19.911198 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":225,"threads_started":1}
I20260810 06:48:20.025331 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:20.024619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.1:34357)
0810 06:48:20.025050 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:20.025138 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":208,"threads_started":1}
I20260810 06:48:20.477975 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:20.477271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.2:53859)
0810 06:48:20.477696 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:20.477799 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":211,"threads_started":1}
I20260810 06:48:20.894860 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 6468
I20260810 06:48:20.911612 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:20.895777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.10.242.1:37171)
0810 06:48:20.896289 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:20.896306 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:20.896353 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:48:20.898200 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:20.898213 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:20.898654 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:20.898942 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:20.900355 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:20.901558 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:20.902799 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:20.903468 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:20.903619 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:20.906083 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:20.906091 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:20.906105 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:20.906141 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:20.908792 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:20.909417 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:20.909423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:20.909429 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:20.909585 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:20.909872 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:20.909875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:20.909877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:20.910421 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:20.910649 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:20.910963 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 06:48:20.911331 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":236,"threads_started":1}
I20260810 06:48:20.912894 6748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:20.913164 7272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:20.912437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:40091 (local address 127.10.242.3:44935)
0810 06:48:20.912877 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:20.912963 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.62:40091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":230,"threads_started":1}
I20260810 06:48:20.996739 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 6468
I20260810 06:48:21.028441 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 6761
I20260810 06:48:21.045492 7279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:21.029392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50108 (local address 127.10.242.2:37055)
0810 06:48:21.030021 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:21.030037 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:21.030078 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:48:21.031672 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:21.031690 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:21.032108 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:21.032398 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:21.033726 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:21.035047 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:21.036396 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:21.037042 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:21.037179 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:21.039707 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:21.039715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:21.039729 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:21.039771 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:21.042684 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:21.043342 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:21.043348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:21.043355 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:21.043526 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:21.043846 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:21.043848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:21.043851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:21.044330 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:21.044562 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:21.044896 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 06:48:21.045215 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":292,"threads_started":1}
I20260810 06:48:21.046849 6976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:21.131232 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 6761
I20260810 06:48:21.151775 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 6994
I20260810 06:48:21.168944 7281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:21.152845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.10.242.3:42007)
0810 06:48:21.153548 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:21.153562 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:21.153609 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:48:21.155292 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:21.155303 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:21.155743 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:21.156048 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:21.157360 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:21.158635 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:21.159817 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:21.160490 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:21.160617 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:21.163155 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:21.163163 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:21.163177 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:21.163221 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:21.166023 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:21.166681 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:21.166688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:21.166694 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:21.166858 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:21.167155 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:21.167158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:21.167160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:21.167644 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:21.167868 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:21.168228 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 06:48:21.168646 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":361,"threads_started":1}
I20260810 06:48:21.170333 7194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:21.252830 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 6994
2026-08-10T06:48:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64359 ms)
[----------] 30 tests from SecurityITest (294730 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:21 dist-test-slave-zj8q krb5kdc[7287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:21 dist-test-slave-zj8q krb5kdc[7287](info): set up 2 sockets
Aug 10 06:48:21 dist-test-slave-zj8q krb5kdc[7287](info): commencing operation
krb5kdc: starting...
W20260810 06:48:22.501101 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:22 dist-test-slave-zj8q krb5kdc[7287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:48:22.772868 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:42521
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:42635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:42521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:23.103370 7304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:23.103633 7304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:23.103766 7304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:23.110869 7304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:23.110975 7304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:23.111052 7304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:23.111123 7304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:23.111191 7304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:23.120772 7304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:42635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:42521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:42521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:23.121991 7304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:23.123319 7304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:23.135249 7309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:23.135313 7312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:23.135221 7310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:23.136036 7304 server_base.cc:1034] running on GCE node
I20260810 06:48:23.136854 7304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:23.138298 7304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:23.139508 7304 hybrid_clock.cc:648] HybridClock initialized: now 1786344503139451 us; error 45 us; skew 500 ppm
I20260810 06:48:23.139846 7304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:23 dist-test-slave-zj8q krb5kdc[7287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344503, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:23.142848 7304 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:48:23.144263 7304 webserver.cc:460] Webserver started at http://127.10.242.62:33545/ using document root <none> and password file <none>
I20260810 06:48:23.144948 7304 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:23.145073 7304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:23.145362 7304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:23.148372 7304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "b794bd5789e4445b85c00357cbd2c011"
format_stamp: "Formatted at 2026-08-10 06:48:23 on dist-test-slave-zj8q"
server_key: "6e43784766c429c7afb4cab2710c447d"
server_key_iv: "55690f9142ce1561d47b4e8b3920ea06"
server_key_version: "encryptionkey@0"
I20260810 06:48:23.149132 7304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "b794bd5789e4445b85c00357cbd2c011"
format_stamp: "Formatted at 2026-08-10 06:48:23 on dist-test-slave-zj8q"
server_key: "6e43784766c429c7afb4cab2710c447d"
server_key_iv: "55690f9142ce1561d47b4e8b3920ea06"
server_key_version: "encryptionkey@0"
I20260810 06:48:23.154341 7304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 06:48:23.157752 7320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:23.159044 7304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:48:23.159209 7304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "b794bd5789e4445b85c00357cbd2c011"
format_stamp: "Formatted at 2026-08-10 06:48:23 on dist-test-slave-zj8q"
server_key: "6e43784766c429c7afb4cab2710c447d"
server_key_iv: "55690f9142ce1561d47b4e8b3920ea06"
server_key_version: "encryptionkey@0"
I20260810 06:48:23.159394 7304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:23.174247 7304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:23.177790 7304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:23.178036 7304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:23.195291 7304 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:42521
I20260810 06:48:23.195314 7414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:42521 every 8 connection(s)
I20260810 06:48:23.196574 7304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:23.199128 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 7304
I20260810 06:48:23.199333 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:23.200016 11208 external_mini_cluster.cc:1428] Setting key 4469526d4cee03ed859ee0985b266e57
I20260810 06:48:23.200287 7417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:23.213994 7417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011: Bootstrap starting.
Aug 10 06:48:23 dist-test-slave-zj8q krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:23.217604 7417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:23.218729 7417 log.cc:824] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:23.221635 7417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011: No bootstrap required, opened a new log
I20260810 06:48:23.222930 7424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:23.201907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.10.242.62:42521)
0810 06:48:23.202734 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:23.202748 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:23.202806 (+ 58us) server_negotiation.cc:407] Connection header received
0810 06:48:23.205014 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:23.205042 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:23.205545 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:23.205960 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:23.207277 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:23.209243 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:23.210583 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:23.211680 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:23.211881 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:23.215906 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:23.215918 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:23.215935 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:23.215977 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:23.219197 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:23.219885 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:23.219891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:23.219896 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:23.220054 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:23.220442 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:23.220445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:23.220447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:23.220916 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:23.221130 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:23.221601 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 06:48:23.221967 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":360,"threads_started":1}
I20260810 06:48:23.229058 7417 raft_consensus.cc:348] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794bd5789e4445b85c00357cbd2c011" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42521 } }
I20260810 06:48:23.229393 7417 raft_consensus.cc:374] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:23.229485 7417 raft_consensus.cc:729] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b794bd5789e4445b85c00357cbd2c011, State: Initialized, Role: FOLLOWER
I20260810 06:48:23.230033 7417 consensus_queue.cc:260] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794bd5789e4445b85c00357cbd2c011" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42521 } }
I20260810 06:48:23.230309 7417 raft_consensus.cc:388] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:23.230439 7417 raft_consensus.cc:482] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:23.230597 7417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:23.232612 7417 raft_consensus.cc:504] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794bd5789e4445b85c00357cbd2c011" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42521 } }
I20260810 06:48:23.233063 7417 leader_election.cc:302] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b794bd5789e4445b85c00357cbd2c011; no voters:
I20260810 06:48:23.233606 7417 leader_election.cc:288] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:23.233706 7426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:23.234645 7426 raft_consensus.cc:686] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [term 1 LEADER]: Becoming Leader. State: Replica: b794bd5789e4445b85c00357cbd2c011, State: Running, Role: LEADER
I20260810 06:48:23.235154 7426 consensus_queue.cc:237] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794bd5789e4445b85c00357cbd2c011" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42521 } }
I20260810 06:48:23.235599 7417 sys_catalog.cc:564] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:23.237720 7431 sys_catalog.cc:455] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b794bd5789e4445b85c00357cbd2c011. Latest consensus state: current_term: 1 leader_uuid: "b794bd5789e4445b85c00357cbd2c011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794bd5789e4445b85c00357cbd2c011" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42521 } } }
I20260810 06:48:23.238277 7431 sys_catalog.cc:458] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:23.237742 7430 sys_catalog.cc:455] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b794bd5789e4445b85c00357cbd2c011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794bd5789e4445b85c00357cbd2c011" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 42521 } } }
I20260810 06:48:23.238909 7430 sys_catalog.cc:458] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:23.243188 7445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:23.248430 7445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:23.256812 7445 catalog_manager.cc:1347] Generated new cluster ID: bdf3175dd6644152aee74b11ec4d283a
I20260810 06:48:23.256913 7445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:23.272390 7445 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:23.273600 7445 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:23.283074 7445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011: Generated new TSK 0
I20260810 06:48:23.283789 7445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:48:23.361404 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42521
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:42635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:23.696033 7470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:23.696293 7470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:23.696398 7470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:23.703348 7470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:23.703444 7470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:23.703564 7470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:48:23.712762 7470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:42635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:23.713856 7470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:23.715144 7470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:23.727536 7476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:23.727572 7477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:23.727607 7480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:23.728224 7470 server_base.cc:1034] running on GCE node
I20260810 06:48:23.728814 7470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:23.729660 7470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:23.730860 7470 hybrid_clock.cc:648] HybridClock initialized: now 1786344503730813 us; error 36 us; skew 500 ppm
I20260810 06:48:23.731148 7470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:23 dist-test-slave-zj8q krb5kdc[7287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344503, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:23.734143 7470 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:48:23.735551 7470 webserver.cc:460] Webserver started at http://127.10.242.1:36943/ using document root <none> and password file <none>
I20260810 06:48:23.736198 7470 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:23.736320 7470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:23.736644 7470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:23.739498 7470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "c87a782954bd4ba782ddaba3cab9e133"
format_stamp: "Formatted at 2026-08-10 06:48:23 on dist-test-slave-zj8q"
server_key: "5829bc1fb19760e404adb29cc2b8bb82"
server_key_iv: "ea4c2b73bf8016a94427da9fe75101fe"
server_key_version: "encryptionkey@0"
I20260810 06:48:23.740232 7470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "c87a782954bd4ba782ddaba3cab9e133"
format_stamp: "Formatted at 2026-08-10 06:48:23 on dist-test-slave-zj8q"
server_key: "5829bc1fb19760e404adb29cc2b8bb82"
server_key_iv: "ea4c2b73bf8016a94427da9fe75101fe"
server_key_version: "encryptionkey@0"
I20260810 06:48:23.745671 7470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:48:23.748996 7489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:23.750295 7470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:48:23.750448 7470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "c87a782954bd4ba782ddaba3cab9e133"
format_stamp: "Formatted at 2026-08-10 06:48:23 on dist-test-slave-zj8q"
server_key: "5829bc1fb19760e404adb29cc2b8bb82"
server_key_iv: "ea4c2b73bf8016a94427da9fe75101fe"
server_key_version: "encryptionkey@0"
I20260810 06:48:23.750631 7470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:23.766497 7470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:23.770479 7470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:23.770738 7470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:23.771633 7470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:23.772938 7470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:23.773016 7470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:23.773136 7470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:23.773190 7470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:23.803038 7470 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:38275
I20260810 06:48:23.803099 7693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:38275 every 8 connection(s)
I20260810 06:48:23.804318 7470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:23.813000 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 7470
I20260810 06:48:23.813202 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:48:23.813851 11208 external_mini_cluster.cc:1428] Setting key 720396359bbd4ace2e8798b6e89291a8
Aug 10 06:48:23 dist-test-slave-zj8q krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344503, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:23.824529 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:23.806758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:60385 (local address 127.10.242.62:42521)
0810 06:48:23.807184 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:23.807188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:23.808220 (+ 1032us) server_negotiation.cc:407] Connection header received
0810 06:48:23.810033 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:23.810038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:23.810131 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:23.810255 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:23.811813 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:23.812554 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:23.814309 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:23.814850 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:23.814985 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:23.818570 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:23.818578 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:23.818583 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:23.818632 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:23.821471 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:23.822199 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:23.822204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:23.822207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:23.822349 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:23.822757 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:23.822761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:23.822764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:23.823110 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:23.823250 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:23.823951 (+ 701us) server_negotiation.cc:299] Negotiation successful
0810 06:48:23.824179 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":213,"threads_started":1}
I20260810 06:48:23.825368 7699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:23.807333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42521 (local address 127.10.242.1:60385)
0810 06:48:23.808048 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:23.808093 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:23.809760 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:23.810506 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:23.810514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:23.811050 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:23.811632 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:23.811650 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:23.812740 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:23.812745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:23.814108 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:23.814121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:23.815091 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:23.815099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:23.815417 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:23.816104 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:23.816127 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:23.818339 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:23.821603 (+ 3264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:23.821610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:23.821624 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:23.822048 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:23.822448 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:23.822451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:23.822453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:23.822656 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:23.823227 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:23.823232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:23.823575 (+ 343us) client_negotiation.cc:769] Sending connection context
0810 06:48:23.823810 (+ 235us) client_negotiation.cc:240] Negotiation successful
0810 06:48:23.824227 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":276,"threads_started":1}
I20260810 06:48:23.826953 7694 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42521
I20260810 06:48:23.827294 7694 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:23.828183 7694 heartbeater.cc:511] Master 127.10.242.62:42521 requested a full tablet report, sending...
I20260810 06:48:23.830416 7357 ts_manager.cc:194] Registered new tserver with Master: c87a782954bd4ba782ddaba3cab9e133 (127.10.242.1:38275)
I20260810 06:48:23.831873 7357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:60385
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:48:23.879215 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42521
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:42635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:24.203430 7706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:24.203701 7706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:24.203848 7706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:24.210881 7706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:24.211000 7706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:24.211153 7706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:48:24.220340 7706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:42635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:24.221517 7706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:24.222788 7706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:24.235031 7711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:24.235004 7712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:24.235062 7714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:24.235927 7706 server_base.cc:1034] running on GCE node
I20260810 06:48:24.236503 7706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:24.237356 7706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:24.238544 7706 hybrid_clock.cc:648] HybridClock initialized: now 1786344504238500 us; error 31 us; skew 500 ppm
I20260810 06:48:24.238847 7706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:24 dist-test-slave-zj8q krb5kdc[7287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344504, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:24.241807 7706 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:48:24.243234 7706 webserver.cc:460] Webserver started at http://127.10.242.2:40109/ using document root <none> and password file <none>
I20260810 06:48:24.243880 7706 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:24.243999 7706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:24.244315 7706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:24.247260 7706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "09d958c0fb3a42599d11274d42781a2f"
format_stamp: "Formatted at 2026-08-10 06:48:24 on dist-test-slave-zj8q"
server_key: "d184ef5a23576c5d40a24de108994ab6"
server_key_iv: "aaf08ee17df5bd41d5300db5d2e62cbe"
server_key_version: "encryptionkey@0"
I20260810 06:48:24.247999 7706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "09d958c0fb3a42599d11274d42781a2f"
format_stamp: "Formatted at 2026-08-10 06:48:24 on dist-test-slave-zj8q"
server_key: "d184ef5a23576c5d40a24de108994ab6"
server_key_iv: "aaf08ee17df5bd41d5300db5d2e62cbe"
server_key_version: "encryptionkey@0"
I20260810 06:48:24.252948 7706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:48:24.256254 7724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:24.257444 7706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:48:24.257592 7706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "09d958c0fb3a42599d11274d42781a2f"
format_stamp: "Formatted at 2026-08-10 06:48:24 on dist-test-slave-zj8q"
server_key: "d184ef5a23576c5d40a24de108994ab6"
server_key_iv: "aaf08ee17df5bd41d5300db5d2e62cbe"
server_key_version: "encryptionkey@0"
I20260810 06:48:24.257783 7706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:24.282589 7706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:24.286661 7706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:24.286939 7706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:24.287844 7706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:24.289115 7706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:24.289199 7706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:24.289315 7706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:24.289367 7706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:24.320161 7706 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:41065
I20260810 06:48:24.320219 7919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:41065 every 8 connection(s)
I20260810 06:48:24.321401 7706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:48:24.323370 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 7706
I20260810 06:48:24.323578 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:48:24.324204 11208 external_mini_cluster.cc:1428] Setting key fbaec570097d46776a8867cb22b3609c
Aug 10 06:48:24 dist-test-slave-zj8q krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344504, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:24.342486 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.324103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:51709 (local address 127.10.242.62:42521)
0810 06:48:24.324280 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:24.324287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:24.325780 (+ 1493us) server_negotiation.cc:407] Connection header received
0810 06:48:24.328454 (+ 2674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:24.328459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:24.328524 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:24.328608 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:24.330485 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.331205 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.332766 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.333192 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.333301 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.336897 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:24.336903 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:24.336906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:24.336965 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:24.339564 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:24.340252 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:24.340257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:24.340261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:24.340415 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:24.340809 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:24.340814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:24.340816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:24.341115 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:24.341263 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:24.341887 (+ 624us) server_negotiation.cc:299] Negotiation successful
0810 06:48:24.342103 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:48:24.343209 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.324809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42521 (local address 127.10.242.2:51709)
0810 06:48:24.325618 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:24.325665 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:24.328166 (+ 2501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:24.328865 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:24.328875 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:24.329565 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:24.330336 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.330362 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.331366 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.331370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.332607 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.332630 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.333408 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.333415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.333642 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.334296 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:24.334318 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:24.336678 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:24.339710 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:24.339716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:24.339727 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:24.340132 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:24.340514 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:24.340516 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:24.340519 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:24.340696 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:24.341244 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:24.341250 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:24.341529 (+ 279us) client_negotiation.cc:769] Sending connection context
0810 06:48:24.341746 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 06:48:24.342102 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":315,"threads_started":1}
I20260810 06:48:24.344566 7921 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42521
I20260810 06:48:24.344861 7921 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:24.345572 7921 heartbeater.cc:511] Master 127.10.242.62:42521 requested a full tablet report, sending...
I20260810 06:48:24.346910 7357 ts_manager.cc:194] Registered new tserver with Master: 09d958c0fb3a42599d11274d42781a2f (127.10.242.2:41065)
I20260810 06:48:24.347692 7357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:51709
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:48:24.386915 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42521
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:42635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:24.713570 7932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:24.713835 7932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:24.713966 7932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:24.720923 7932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:24.721019 7932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:24.721165 7932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:48:24.730407 7932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:42635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:24.731525 7932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:24.732769 7932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:24.745337 7941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:24.745303 7938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:24.745563 7937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:24.746420 7932 server_base.cc:1034] running on GCE node
I20260810 06:48:24.746973 7932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:24.747814 7932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:24.749027 7932 hybrid_clock.cc:648] HybridClock initialized: now 1786344504748972 us; error 40 us; skew 500 ppm
I20260810 06:48:24.749322 7932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:24 dist-test-slave-zj8q krb5kdc[7287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344504, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:24.756614 7932 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:48:24.758044 7932 webserver.cc:460] Webserver started at http://127.10.242.3:42551/ using document root <none> and password file <none>
I20260810 06:48:24.758800 7932 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:24.758910 7932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:24.759236 7932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:24.762272 7932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "068ad543543f465fb3b57502b7bf7b24"
format_stamp: "Formatted at 2026-08-10 06:48:24 on dist-test-slave-zj8q"
server_key: "9c6b84939ba47d4ccc272cfc53d8ef57"
server_key_iv: "da8ee636529b514909f68db5806e95fb"
server_key_version: "encryptionkey@0"
I20260810 06:48:24.763134 7932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "068ad543543f465fb3b57502b7bf7b24"
format_stamp: "Formatted at 2026-08-10 06:48:24 on dist-test-slave-zj8q"
server_key: "9c6b84939ba47d4ccc272cfc53d8ef57"
server_key_iv: "da8ee636529b514909f68db5806e95fb"
server_key_version: "encryptionkey@0"
I20260810 06:48:24.768311 7932 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 06:48:24.771699 7954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:24.772933 7932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:48:24.773074 7932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "068ad543543f465fb3b57502b7bf7b24"
format_stamp: "Formatted at 2026-08-10 06:48:24 on dist-test-slave-zj8q"
server_key: "9c6b84939ba47d4ccc272cfc53d8ef57"
server_key_iv: "da8ee636529b514909f68db5806e95fb"
server_key_version: "encryptionkey@0"
I20260810 06:48:24.773272 7932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:24.800881 7932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:24.805573 7932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:24.805895 7932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:24.807061 7932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:24.808679 7932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:24.808772 7932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:24.808892 7932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:24.808959 7932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:24.834811 7694 heartbeater.cc:503] Master 127.10.242.62:42521 was elected leader, sending a full tablet report...
I20260810 06:48:24.839416 7932 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:35199
I20260810 06:48:24.839458 8149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:35199 every 8 connection(s)
I20260810 06:48:24.840624 7932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:48:24.842335 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 7932
I20260810 06:48:24.842530 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:48:24.842983 11208 external_mini_cluster.cc:1428] Setting key b641aeb9b18e5766e60d06d679f2c57d
Aug 10 06:48:24 dist-test-slave-zj8q krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344504, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:24.859496 8156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.843331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:48575 (local address 127.10.242.62:42521)
0810 06:48:24.844119 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:24.844125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:24.844858 (+ 733us) server_negotiation.cc:407] Connection header received
0810 06:48:24.846875 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:24.846880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:24.846951 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:24.847031 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:24.848367 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.849118 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.850648 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.851048 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.851136 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.854145 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:24.854171 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:24.854191 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:24.854229 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:24.856654 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:24.857404 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:24.857410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:24.857413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:24.857552 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:24.858024 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:24.858028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:24.858031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:24.858349 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:24.858456 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:24.859034 (+ 578us) server_negotiation.cc:299] Negotiation successful
0810 06:48:24.859218 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":526,"threads_started":1}
I20260810 06:48:24.860392 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.843886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:42521 (local address 127.10.242.3:48575)
0810 06:48:24.844708 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:24.844752 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:24.846635 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:24.847197 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:24.847204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:24.847703 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:24.848240 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.848257 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.849260 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.849264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.850535 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.850549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.851213 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.851218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.851410 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.851962 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:24.851982 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:24.853970 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:24.856778 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:24.856786 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:24.856800 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:24.857302 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:24.857669 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:24.857673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:24.857676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:24.857916 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:24.858454 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:24.858460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:24.858750 (+ 290us) client_negotiation.cc:769] Sending connection context
0810 06:48:24.858961 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 06:48:24.859350 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":343,"threads_started":1}
I20260810 06:48:24.861726 8152 heartbeater.cc:347] Connected to a master server at 127.10.242.62:42521
I20260810 06:48:24.862006 8152 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:24.862758 8152 heartbeater.cc:511] Master 127.10.242.62:42521 requested a full tablet report, sending...
I20260810 06:48:24.864100 7357 ts_manager.cc:194] Registered new tserver with Master: 068ad543543f465fb3b57502b7bf7b24 (127.10.242.3:35199)
I20260810 06:48:24.864948 7357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:48575
I20260810 06:48:24.870014 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:48:24.886142 8156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.874277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53568 (local address 127.10.242.62:42521)
0810 06:48:24.874421 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:24.874424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:24.874870 (+ 446us) server_negotiation.cc:407] Connection header received
0810 06:48:24.875017 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:24.875022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:24.875096 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:24.875173 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:24.876636 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.877382 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.878744 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.879143 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.879254 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.881469 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:24.881473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:24.881476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:24.881504 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:24.883676 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:24.884324 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:24.884329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:24.884333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:24.884443 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:24.884809 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:24.884815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:24.884819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:24.885104 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:24.885327 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:24.885650 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 06:48:24.885862 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:48:24.889595 7357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:48:24.896350 7357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:48:24.927799 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.907818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.10.242.2:41065)
0810 06:48:24.919546 (+ 11728us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:24.919556 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:24.920157 (+ 601us) server_negotiation.cc:407] Connection header received
0810 06:48:24.920264 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:24.920271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:24.920497 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:24.920693 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:24.921552 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.923071 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.925585 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.926743 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.926872 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.927063 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:24.927336 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 06:48:24.927493 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11585,"thread_start_us":11473,"threads_started":1}
I20260810 06:48:24.927881 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.907636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:41065 (local address 127.0.0.1:60042)
0810 06:48:24.919881 (+ 12245us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:24.919898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:24.920022 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:24.920944 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:24.920949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:24.920974 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:24.921360 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.921373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.923230 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.923235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.925431 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.925443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.926935 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.926941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.927122 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.927172 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 06:48:24.927275 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 06:48:24.927455 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12112,"mutex_wait_us":52,"thread_start_us":10641,"threads_started":1}
I20260810 06:48:24.928634 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.907591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.10.242.3:35199)
0810 06:48:24.917951 (+ 10360us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:24.917960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:24.918982 (+ 1022us) server_negotiation.cc:407] Connection header received
0810 06:48:24.919107 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:24.919115 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:24.919367 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:24.919569 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:24.920342 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.921873 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.925946 (+ 4073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.927283 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.927399 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.927601 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:24.928118 (+ 517us) server_negotiation.cc:299] Negotiation successful
0810 06:48:24.928303 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10174,"thread_start_us":10348,"threads_started":1}
I20260810 06:48:24.929360 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.907274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:35199 (local address 127.0.0.1:51106)
0810 06:48:24.918685 (+ 11411us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:24.918738 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:24.918903 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:24.919682 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:24.919689 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:24.919723 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:24.920145 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.920158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.922043 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.922048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.925762 (+ 3714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.925778 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.927747 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.927753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.927909 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.927942 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:48:24.928817 (+ 875us) client_negotiation.cc:240] Negotiation successful
0810 06:48:24.928954 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11152,"mutex_wait_us":1,"thread_start_us":12321,"threads_started":1}
I20260810 06:48:24.932451 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.920008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:38275 (local address 127.0.0.1:35128)
0810 06:48:24.925062 (+ 5054us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:24.925078 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:24.925197 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:24.925771 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:24.925775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:24.925801 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:24.926146 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.926182 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.927990 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.927995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.929960 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:24.929976 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.931723 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.931730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:24.931889 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.931920 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:48:24.932016 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:48:24.932132 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4935,"thread_start_us":4794,"threads_started":1}
I20260810 06:48:24.931957 7807 tablet_service.cc:1467] Processing CreateTablet for tablet 3534ff8d44bc46b19bb320ccb9f49b20 (DEFAULT_TABLE table=test-table [id=c0959a2f0e9a47b693423f12abcc42b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:24.932528 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:24.920317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35128 (local address 127.10.242.1:38275)
0810 06:48:24.924654 (+ 4337us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:24.924662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:24.925178 (+ 516us) server_negotiation.cc:407] Connection header received
0810 06:48:24.925367 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:24.925375 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:24.925580 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:24.925771 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:24.926355 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.927837 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.930119 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:24.931508 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:24.931648 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:24.931863 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:24.932073 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 06:48:24.932239 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4175,"thread_start_us":4006,"threads_started":1}
I20260810 06:48:24.932664 8037 tablet_service.cc:1467] Processing CreateTablet for tablet 3534ff8d44bc46b19bb320ccb9f49b20 (DEFAULT_TABLE table=test-table [id=c0959a2f0e9a47b693423f12abcc42b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:24.933307 7807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3534ff8d44bc46b19bb320ccb9f49b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:24.933940 8037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3534ff8d44bc46b19bb320ccb9f49b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:24.936571 7566 tablet_service.cc:1467] Processing CreateTablet for tablet 3534ff8d44bc46b19bb320ccb9f49b20 (DEFAULT_TABLE table=test-table [id=c0959a2f0e9a47b693423f12abcc42b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:24.937907 7566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3534ff8d44bc46b19bb320ccb9f49b20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:24.945785 8186 tablet_bootstrap.cc:492] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: Bootstrap starting.
I20260810 06:48:24.947870 8189 tablet_bootstrap.cc:492] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: Bootstrap starting.
I20260810 06:48:24.948921 8186 tablet_bootstrap.cc:654] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:24.950214 8186 log.cc:824] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:24.951133 8191 tablet_bootstrap.cc:492] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: Bootstrap starting.
I20260810 06:48:24.951301 8189 tablet_bootstrap.cc:654] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:24.952589 8189 log.cc:824] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:24.954087 8191 tablet_bootstrap.cc:654] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:24.955312 8186 tablet_bootstrap.cc:492] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: No bootstrap required, opened a new log
I20260810 06:48:24.955307 8191 log.cc:824] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:24.955574 8186 ts_tablet_manager.cc:1397] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 06:48:24.955643 8189 tablet_bootstrap.cc:492] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: No bootstrap required, opened a new log
I20260810 06:48:24.955897 8189 ts_tablet_manager.cc:1397] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 06:48:24.958441 8191 tablet_bootstrap.cc:492] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: No bootstrap required, opened a new log
I20260810 06:48:24.958701 8191 ts_tablet_manager.cc:1397] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260810 06:48:24.963157 8186 raft_consensus.cc:348] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } }
I20260810 06:48:24.963488 8186 raft_consensus.cc:374] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:24.963584 8186 raft_consensus.cc:729] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d958c0fb3a42599d11274d42781a2f, State: Initialized, Role: FOLLOWER
I20260810 06:48:24.964218 8186 consensus_queue.cc:260] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } }
I20260810 06:48:24.965206 7921 heartbeater.cc:503] Master 127.10.242.62:42521 was elected leader, sending a full tablet report...
I20260810 06:48:24.965741 8186 ts_tablet_manager.cc:1428] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260810 06:48:24.965816 8189 raft_consensus.cc:348] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } }
I20260810 06:48:24.966259 8189 raft_consensus.cc:374] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:24.966355 8189 raft_consensus.cc:729] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068ad543543f465fb3b57502b7bf7b24, State: Initialized, Role: FOLLOWER
I20260810 06:48:24.966900 8189 consensus_queue.cc:260] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } }
I20260810 06:48:24.967938 8152 heartbeater.cc:503] Master 127.10.242.62:42521 was elected leader, sending a full tablet report...
I20260810 06:48:24.968497 8189 ts_tablet_manager.cc:1428] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: Time spent starting tablet: real 0.012s user 0.006s sys 0.007s
I20260810 06:48:24.969306 8191 raft_consensus.cc:348] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } }
I20260810 06:48:24.969764 8191 raft_consensus.cc:374] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:24.969909 8191 raft_consensus.cc:729] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87a782954bd4ba782ddaba3cab9e133, State: Initialized, Role: FOLLOWER
I20260810 06:48:24.970432 8191 consensus_queue.cc:260] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } }
I20260810 06:48:24.973095 8191 ts_tablet_manager.cc:1428] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
W20260810 06:48:25.056633 7697 tablet.cc:2369] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:48:25.073279 7924 tablet.cc:2369] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:48:25.092159 8153 tablet.cc:2369] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:25.328318 8197 raft_consensus.cc:482] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:25.328599 8197 raft_consensus.cc:504] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } }
I20260810 06:48:25.330128 8197 leader_election.cc:288] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09d958c0fb3a42599d11274d42781a2f (127.10.242.2:41065), c87a782954bd4ba782ddaba3cab9e133 (127.10.242.1:38275)
I20260810 06:48:25.336647 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:25.330681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:44587 (local address 127.10.242.1:38275)
0810 06:48:25.330903 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:25.330910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:25.330931 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:48:25.331009 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:25.331014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:25.331088 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:25.331186 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:25.331632 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.332528 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.334612 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.335740 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.335863 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:25.335917 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:25.336246 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 06:48:25.336362 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 06:48:25.336669 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:25.330442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:38275 (local address 127.10.242.3:44587)
0810 06:48:25.330709 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:25.330730 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:25.330853 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:25.331238 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:25.331242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:25.331264 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:25.331473 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:25.331485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.332666 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.332671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:25.334438 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:25.334450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.335888 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.335894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:25.336081 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:25.336117 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:48:25.336222 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 06:48:25.336414 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":29}
I20260810 06:48:25.337229 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:25.330592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:55985 (local address 127.10.242.2:41065)
0810 06:48:25.330789 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:25.330795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:25.331192 (+ 397us) server_negotiation.cc:407] Connection header received
0810 06:48:25.331350 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:25.331354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:25.331422 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:25.331512 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:25.332091 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.332880 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.335310 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.336252 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.336382 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:25.336446 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:25.336849 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 06:48:25.336978 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:48:25.337760 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:25.330528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:41065 (local address 127.10.242.3:55985)
0810 06:48:25.331088 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:25.331105 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:25.331230 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:25.331556 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:25.331561 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:25.331585 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:25.331955 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:25.331968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.333021 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.333025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:25.335140 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:25.335156 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.336443 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.336449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:25.336651 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:25.336688 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:48:25.337395 (+ 707us) client_negotiation.cc:240] Negotiation successful
0810 06:48:25.337500 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":16,"thread_start_us":759,"threads_started":1}
I20260810 06:48:25.337687 7611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3534ff8d44bc46b19bb320ccb9f49b20" candidate_uuid: "068ad543543f465fb3b57502b7bf7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87a782954bd4ba782ddaba3cab9e133" is_pre_election: true
I20260810 06:48:25.338234 7611 raft_consensus.cc:2454] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068ad543543f465fb3b57502b7bf7b24 in term 0.
I20260810 06:48:25.338723 7843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3534ff8d44bc46b19bb320ccb9f49b20" candidate_uuid: "068ad543543f465fb3b57502b7bf7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d958c0fb3a42599d11274d42781a2f" is_pre_election: true
I20260810 06:48:25.338948 7959 leader_election.cc:302] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068ad543543f465fb3b57502b7bf7b24, c87a782954bd4ba782ddaba3cab9e133; no voters:
I20260810 06:48:25.339314 8197 raft_consensus.cc:2781] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:48:25.339265 7843 raft_consensus.cc:2454] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068ad543543f465fb3b57502b7bf7b24 in term 0.
I20260810 06:48:25.339444 8197 raft_consensus.cc:482] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:48:25.339540 8197 raft_consensus.cc:3037] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:25.341974 8197 raft_consensus.cc:504] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } }
I20260810 06:48:25.342532 8197 leader_election.cc:288] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [CANDIDATE]: Term 1 election: Requested vote from peers 09d958c0fb3a42599d11274d42781a2f (127.10.242.2:41065), c87a782954bd4ba782ddaba3cab9e133 (127.10.242.1:38275)
I20260810 06:48:25.342917 7843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3534ff8d44bc46b19bb320ccb9f49b20" candidate_uuid: "068ad543543f465fb3b57502b7bf7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d958c0fb3a42599d11274d42781a2f"
I20260810 06:48:25.342995 7611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3534ff8d44bc46b19bb320ccb9f49b20" candidate_uuid: "068ad543543f465fb3b57502b7bf7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87a782954bd4ba782ddaba3cab9e133"
I20260810 06:48:25.343082 7843 raft_consensus.cc:3037] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:25.343209 7611 raft_consensus.cc:3037] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:25.345247 7843 raft_consensus.cc:2454] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068ad543543f465fb3b57502b7bf7b24 in term 1.
I20260810 06:48:25.345490 7611 raft_consensus.cc:2454] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068ad543543f465fb3b57502b7bf7b24 in term 1.
I20260810 06:48:25.345616 7957 leader_election.cc:302] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068ad543543f465fb3b57502b7bf7b24, 09d958c0fb3a42599d11274d42781a2f; no voters:
I20260810 06:48:25.345908 8197 raft_consensus.cc:2781] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:25.346371 8197 raft_consensus.cc:686] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 1 LEADER]: Becoming Leader. State: Replica: 068ad543543f465fb3b57502b7bf7b24, State: Running, Role: LEADER
I20260810 06:48:25.346832 8197 consensus_queue.cc:237] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } }
I20260810 06:48:25.351107 7355 catalog_manager.cc:5495] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 068ad543543f465fb3b57502b7bf7b24 (127.10.242.3). New cstate: current_term: 1 leader_uuid: "068ad543543f465fb3b57502b7bf7b24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068ad543543f465fb3b57502b7bf7b24" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 35199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 } health_report { overall_health: UNKNOWN } } }
I20260810 06:48:25.368705 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:25.363971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.10.242.3:35199)
0810 06:48:25.364158 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:25.364162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:25.364178 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:48:25.364336 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:25.364339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:25.364412 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:25.364525 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:48:25.364822 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.365530 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.366925 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.367333 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.367426 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:25.367795 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:48:25.368253 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:48:25.368353 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:25.368407 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 06:48:25.368467 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:48:25.376190 7843 raft_consensus.cc:1261] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [term 1 FOLLOWER]: Refusing update from remote peer 068ad543543f465fb3b57502b7bf7b24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:25.376201 7611 raft_consensus.cc:1261] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [term 1 FOLLOWER]: Refusing update from remote peer 068ad543543f465fb3b57502b7bf7b24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:25.377095 8205 consensus_queue.cc:1035] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d958c0fb3a42599d11274d42781a2f" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 41065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:25.377465 8197 consensus_queue.cc:1035] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87a782954bd4ba782ddaba3cab9e133" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 38275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:25.408238 7355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:25.408500 7355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:25.411635 7355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b794bd5789e4445b85c00357cbd2c011: Sending DeleteTablet for 3 replicas of tablet 3534ff8d44bc46b19bb320ccb9f49b20
I20260810 06:48:25.412731 7807 tablet_service.cc:1514] Processing DeleteTablet for tablet 3534ff8d44bc46b19bb320ccb9f49b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:25 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:60042
I20260810 06:48:25.413023 8037 tablet_service.cc:1514] Processing DeleteTablet for tablet 3534ff8d44bc46b19bb320ccb9f49b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:25 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:51106
I20260810 06:48:25.413131 7566 tablet_service.cc:1514] Processing DeleteTablet for tablet 3534ff8d44bc46b19bb320ccb9f49b20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:25 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:35128
I20260810 06:48:25.414745 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 7470
I20260810 06:48:25.425065 8223 tablet_replica.cc:330] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: stopping tablet replica
I20260810 06:48:25.425660 8223 raft_consensus.cc:2229] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:25.426307 8223 raft_consensus.cc:2258] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:25.428623 8223 ts_tablet_manager.cc:1905] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:25.435669 8223 ts_tablet_manager.cc:1918] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:25.435894 8223 log.cc:1196] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/3534ff8d44bc46b19bb320ccb9f49b20
I20260810 06:48:25.436596 8223 ts_tablet_manager.cc:1939] T 3534ff8d44bc46b19bb320ccb9f49b20 P 09d958c0fb3a42599d11274d42781a2f: Deleting consensus metadata
I20260810 06:48:25.438549 7325 catalog_manager.cc:4841] TS 09d958c0fb3a42599d11274d42781a2f (127.10.242.2:41065): tablet 3534ff8d44bc46b19bb320ccb9f49b20 (table test-table [id=c0959a2f0e9a47b693423f12abcc42b6]) successfully deleted
I20260810 06:48:25.442287 8226 tablet_replica.cc:330] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: stopping tablet replica
I20260810 06:48:25.442807 8226 raft_consensus.cc:2229] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:25.443415 8226 raft_consensus.cc:2258] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:25.444330 8224 tablet_replica.cc:330] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: stopping tablet replica
I20260810 06:48:25.444747 8224 raft_consensus.cc:2229] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:48:25.445396 8224 raft_consensus.cc:2258] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:25.447124 8226 ts_tablet_manager.cc:1905] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:25.447984 8224 ts_tablet_manager.cc:1905] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:48:25 dist-test-slave-zj8q krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:48:25.454038 8226 ts_tablet_manager.cc:1918] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:25.454262 8226 log.cc:1196] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/3534ff8d44bc46b19bb320ccb9f49b20
I20260810 06:48:25.454854 8226 ts_tablet_manager.cc:1939] T 3534ff8d44bc46b19bb320ccb9f49b20 P c87a782954bd4ba782ddaba3cab9e133: Deleting consensus metadata
I20260810 06:48:25.455086 8224 ts_tablet_manager.cc:1918] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:25.455281 8224 log.cc:1196] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/3534ff8d44bc46b19bb320ccb9f49b20
I20260810 06:48:25.455823 8224 ts_tablet_manager.cc:1939] T 3534ff8d44bc46b19bb320ccb9f49b20 P 068ad543543f465fb3b57502b7bf7b24: Deleting consensus metadata
I20260810 06:48:25.456678 7329 catalog_manager.cc:4841] TS c87a782954bd4ba782ddaba3cab9e133 (127.10.242.1:38275): tablet 3534ff8d44bc46b19bb320ccb9f49b20 (table test-table [id=c0959a2f0e9a47b693423f12abcc42b6]) successfully deleted
I20260810 06:48:25.457618 7329 catalog_manager.cc:4841] TS 068ad543543f465fb3b57502b7bf7b24 (127.10.242.3:35199): tablet 3534ff8d44bc46b19bb320ccb9f49b20 (table test-table [id=c0959a2f0e9a47b693423f12abcc42b6]) successfully deleted
I20260810 06:48:25.459671 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:25.416326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35142 (local address 127.10.242.1:38275)
0810 06:48:25.416509 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:25.416516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:25.441512 (+ 24996us) server_negotiation.cc:407] Connection header received
0810 06:48:25.441743 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:25.441749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:25.441859 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:25.441941 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:25.443980 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.444840 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.446564 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.447203 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.447301 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:25.452647 (+ 5346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:25.452665 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:25.452682 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:25.452721 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:25.456742 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:25.457625 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:25.457634 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:25.457637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:25.457808 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:25.458216 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:25.458222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:25.458225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:25.458559 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:25.458843 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:25.459146 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 06:48:25.459342 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:48:25.460269 7685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:25.577296 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 7470
I20260810 06:48:25.608687 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 7706
Aug 10 06:48:25 dist-test-slave-zj8q krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:48:25.625043 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:25.609520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.10.242.2:41065)
0810 06:48:25.609685 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:25.609690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:25.609709 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:48:25.609864 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:25.609868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:25.609931 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:25.610009 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:25.611766 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.612481 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.614540 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.614941 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.615036 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:25.619765 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:25.619780 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:25.619796 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:25.619832 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:25.622436 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:25.623081 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:25.623089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:25.623091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:25.623230 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:25.623578 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:25.623581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:25.623583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:25.623945 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:25.624246 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:25.624518 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 06:48:25.624725 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:48:25.625650 7915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:25.713032 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 7706
I20260810 06:48:25.734400 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 7932
Aug 10 06:48:25 dist-test-slave-zj8q krb5kdc[7287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:48:25.753297 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:25.735353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51130 (local address 127.10.242.3:35199)
0810 06:48:25.735516 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:25.735520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:25.735609 (+ 89us) server_negotiation.cc:407] Connection header received
0810 06:48:25.735815 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:25.735819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:25.735889 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:25.735999 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:25.737790 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.738670 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.740457 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:25.740898 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:25.740986 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:25.746291 (+ 5305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:25.746306 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:25.746329 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:25.746377 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:25.750479 (+ 4102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:25.751118 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:25.751129 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:25.751132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:25.751354 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:25.751684 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:25.751689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:25.751693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:25.752148 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:25.752488 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:25.752655 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 06:48:25.752866 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:48:25.754087 8142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:25.890318 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 7932
I20260810 06:48:25.917940 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 7304
I20260810 06:48:25.919075 7410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:26.028155 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 7304
2026-08-10T06:48:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4782 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T06:48:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:26Z Disabled control of system clock
I20260810 06:48:26.118454 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:46189
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:46189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:48:26.445748 8238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:26.446014 8238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:26.446137 8238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:26.453145 8238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:26.453244 8238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:26.453306 8238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:26.453388 8238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:26.453446 8238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:26.462229 8238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:46189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:46189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:26.463572 8238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:26.465287 8238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:26.476423 8244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:26.476351 8245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:26.476663 8248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:26.477402 8238 server_base.cc:1034] running on GCE node
I20260810 06:48:26.478072 8238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:26.479411 8238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:26.480605 8238 hybrid_clock.cc:648] HybridClock initialized: now 1786344506480564 us; error 32 us; skew 500 ppm
I20260810 06:48:26.480921 8238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:26.482987 8238 webserver.cc:460] Webserver started at http://127.10.242.62:40187/ using document root <none> and password file <none>
I20260810 06:48:26.483605 8238 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:26.483700 8238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:26.484001 8238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:26.486852 8238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "49e95663cdbc4c1c9487d8f174d1c000"
format_stamp: "Formatted at 2026-08-10 06:48:26 on dist-test-slave-zj8q"
server_key: "a4c1136acb3b5744c6a99dd95ee5dd03"
server_key_iv: "29226bace3831d9c7d84e344e174600d"
server_key_version: "encryptionkey@0"
I20260810 06:48:26.487653 8238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "49e95663cdbc4c1c9487d8f174d1c000"
format_stamp: "Formatted at 2026-08-10 06:48:26 on dist-test-slave-zj8q"
server_key: "a4c1136acb3b5744c6a99dd95ee5dd03"
server_key_iv: "29226bace3831d9c7d84e344e174600d"
server_key_version: "encryptionkey@0"
I20260810 06:48:26.492383 8238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:48:26.495602 8256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:26.496793 8238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:48:26.496956 8238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "49e95663cdbc4c1c9487d8f174d1c000"
format_stamp: "Formatted at 2026-08-10 06:48:26 on dist-test-slave-zj8q"
server_key: "a4c1136acb3b5744c6a99dd95ee5dd03"
server_key_iv: "29226bace3831d9c7d84e344e174600d"
server_key_version: "encryptionkey@0"
I20260810 06:48:26.497152 8238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:26.521579 8238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:26.522377 8238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:26.522647 8238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:26.539284 8238 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:46189
I20260810 06:48:26.539309 8360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:46189 every 8 connection(s)
I20260810 06:48:26.540548 8238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:26.543030 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 8238
I20260810 06:48:26.543200 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:26.543622 11208 external_mini_cluster.cc:1428] Setting key 8eeb3940e1117d6eec83b7f374cff729
I20260810 06:48:26.544195 8361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:26.558689 8365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:26.545396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48326 (local address 127.10.242.62:46189)
0810 06:48:26.546499 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:26.546513 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:26.546567 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:48:26.548804 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:26.548837 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:26.549384 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:26.549804 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:26.550371 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:26.552334 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:26.553741 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:26.554793 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:26.554982 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:26.556063 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:26.556073 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:26.556090 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:26.556799 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:26.556888 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:26.556913 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:26.557017 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:26.557161 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 06:48:26.557463 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":357,"threads_started":1}
I20260810 06:48:26.558804 8361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000: Bootstrap starting.
I20260810 06:48:26.562287 8361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:26.563380 8361 log.cc:824] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:26.566039 8361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000: No bootstrap required, opened a new log
I20260810 06:48:26.573252 8361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e95663cdbc4c1c9487d8f174d1c000" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46189 } }
I20260810 06:48:26.573582 8361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:26.573652 8361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e95663cdbc4c1c9487d8f174d1c000, State: Initialized, Role: FOLLOWER
I20260810 06:48:26.574111 8361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e95663cdbc4c1c9487d8f174d1c000" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46189 } }
I20260810 06:48:26.574381 8361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:26.574481 8361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:26.574599 8361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:26.576622 8361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e95663cdbc4c1c9487d8f174d1c000" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46189 } }
I20260810 06:48:26.577033 8361 leader_election.cc:302] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49e95663cdbc4c1c9487d8f174d1c000; no voters:
I20260810 06:48:26.577586 8361 leader_election.cc:288] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:26.577742 8367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:26.578424 8367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [term 1 LEADER]: Becoming Leader. State: Replica: 49e95663cdbc4c1c9487d8f174d1c000, State: Running, Role: LEADER
I20260810 06:48:26.579037 8367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e95663cdbc4c1c9487d8f174d1c000" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46189 } }
I20260810 06:48:26.579668 8361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:26.581781 8368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49e95663cdbc4c1c9487d8f174d1c000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e95663cdbc4c1c9487d8f174d1c000" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46189 } } }
I20260810 06:48:26.582257 8368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:26.582252 8369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49e95663cdbc4c1c9487d8f174d1c000. Latest consensus state: current_term: 1 leader_uuid: "49e95663cdbc4c1c9487d8f174d1c000" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e95663cdbc4c1c9487d8f174d1c000" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 46189 } } }
I20260810 06:48:26.582566 8369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:26.584093 8383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:26.588779 8383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:26.596565 8383 catalog_manager.cc:1347] Generated new cluster ID: 831599449c2f4b2d9e3be5e11dba1255
I20260810 06:48:26.596675 8383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:26.611963 8383 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:26.613234 8383 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:26.626117 8383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000: Generated new TSK 0
I20260810 06:48:26.626971 8383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:26.641175 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46189
--builtin_ntp_servers=127.10.242.20:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:48:26.973116 8397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:26.973382 8397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:26.973508 8397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:26.980746 8397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:26.980835 8397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:26.980960 8397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:48:26.989904 8397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:26.991130 8397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:26.992893 8397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:27.005371 8411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:27.005451 8405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:27.005450 8409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:27.006285 8397 server_base.cc:1034] running on GCE node
I20260810 06:48:27.006820 8397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:27.007663 8397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:27.008857 8397 hybrid_clock.cc:648] HybridClock initialized: now 1786344507008807 us; error 44 us; skew 500 ppm
I20260810 06:48:27.009156 8397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:27.011340 8397 webserver.cc:460] Webserver started at http://127.10.242.1:33551/ using document root <none> and password file <none>
I20260810 06:48:27.011979 8397 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:27.012076 8397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:27.012377 8397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:27.015218 8397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "7a96b6d3a0414d748136f5edec95858b"
format_stamp: "Formatted at 2026-08-10 06:48:27 on dist-test-slave-zj8q"
server_key: "8f0d561f5d97dc238fb073214a6218e1"
server_key_iv: "cf1a4255ff1473c238d8112ae26e9bc5"
server_key_version: "encryptionkey@0"
I20260810 06:48:27.015971 8397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "7a96b6d3a0414d748136f5edec95858b"
format_stamp: "Formatted at 2026-08-10 06:48:27 on dist-test-slave-zj8q"
server_key: "8f0d561f5d97dc238fb073214a6218e1"
server_key_iv: "cf1a4255ff1473c238d8112ae26e9bc5"
server_key_version: "encryptionkey@0"
I20260810 06:48:27.020728 8397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:48:27.023963 8418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:27.025139 8397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:48:27.025278 8397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "7a96b6d3a0414d748136f5edec95858b"
format_stamp: "Formatted at 2026-08-10 06:48:27 on dist-test-slave-zj8q"
server_key: "8f0d561f5d97dc238fb073214a6218e1"
server_key_iv: "cf1a4255ff1473c238d8112ae26e9bc5"
server_key_version: "encryptionkey@0"
I20260810 06:48:27.025470 8397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:27.036525 8397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:27.037231 8397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:27.037478 8397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:27.038362 8397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:27.039744 8397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:27.039830 8397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:27.039911 8397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:27.039963 8397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:27.069718 8397 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:45947
I20260810 06:48:27.069761 8618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:45947 every 8 connection(s)
I20260810 06:48:27.071080 8397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:27.080185 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 8397
I20260810 06:48:27.080335 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:48:27.080802 11208 external_mini_cluster.cc:1428] Setting key a5277c3577bdf609a59a590b604832cb
I20260810 06:48:27.083382 8623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:27.073566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:47079 (local address 127.10.242.62:46189)
0810 06:48:27.073995 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:27.073999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:27.075082 (+ 1083us) server_negotiation.cc:407] Connection header received
0810 06:48:27.076826 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:27.076831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:27.076914 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:27.077010 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:27.078058 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:27.078841 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:27.080515 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:27.081047 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:27.081184 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:27.082515 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:27.082520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:27.082525 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:27.082543 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:27.082587 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:27.082599 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:27.082699 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:27.082956 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 06:48:27.083050 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":218,"threads_started":1}
I20260810 06:48:27.083626 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46189
--builtin_ntp_servers=127.10.242.20:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:48:27.084425 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:27.074030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:46189 (local address 127.10.242.1:47079)
0810 06:48:27.074939 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:27.074973 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:27.076560 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:27.077252 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:27.077260 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:27.077351 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:27.077889 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:27.077906 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:27.078997 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:27.079001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:27.080378 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:27.080394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:27.081278 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:27.081287 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:27.081539 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:27.082198 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:48:27.082202 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:27.082260 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:48:27.082335 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:27.082709 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:27.082721 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:27.082728 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:48:27.082925 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 06:48:27.083248 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":297,"threads_started":1}
I20260810 06:48:27.085925 8620 heartbeater.cc:347] Connected to a master server at 127.10.242.62:46189
I20260810 06:48:27.086233 8620 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:27.086931 8620 heartbeater.cc:511] Master 127.10.242.62:46189 requested a full tablet report, sending...
I20260810 06:48:27.089035 8291 ts_manager.cc:194] Registered new tserver with Master: 7a96b6d3a0414d748136f5edec95858b (127.10.242.1:45947)
I20260810 06:48:27.090950 8291 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.1:47079
W20260810 06:48:27.589190 8626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:27.589666 8626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:27.589877 8626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:27.601318 8626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:27.601462 8626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:27.601742 8626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:48:27.615975 8626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:27.618013 8626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:27.621012 8626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:27.641943 8635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:27.642076 8636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:27.642000 8639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:27.648002 8626 server_base.cc:1034] running on GCE node
I20260810 06:48:27.648936 8626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:27.650436 8626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:27.651705 8626 hybrid_clock.cc:648] HybridClock initialized: now 1786344507651626 us; error 59 us; skew 500 ppm
I20260810 06:48:27.652223 8626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:27.655660 8626 webserver.cc:460] Webserver started at http://127.10.242.2:33979/ using document root <none> and password file <none>
I20260810 06:48:27.656711 8626 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:27.656864 8626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:27.657330 8626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:27.661927 8626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "b8d17a646bae454fb4bb13ed236be943"
format_stamp: "Formatted at 2026-08-10 06:48:27 on dist-test-slave-zj8q"
server_key: "0963b7b7d317d4d3cbe3aa6a59628176"
server_key_iv: "adbad7296be9b3c397d47b58fafcbdf1"
server_key_version: "encryptionkey@0"
I20260810 06:48:27.663213 8626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "b8d17a646bae454fb4bb13ed236be943"
format_stamp: "Formatted at 2026-08-10 06:48:27 on dist-test-slave-zj8q"
server_key: "0963b7b7d317d4d3cbe3aa6a59628176"
server_key_iv: "adbad7296be9b3c397d47b58fafcbdf1"
server_key_version: "encryptionkey@0"
I20260810 06:48:27.671536 8626 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260810 06:48:27.676865 8653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:27.679088 8626 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 06:48:27.679291 8626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "b8d17a646bae454fb4bb13ed236be943"
format_stamp: "Formatted at 2026-08-10 06:48:27 on dist-test-slave-zj8q"
server_key: "0963b7b7d317d4d3cbe3aa6a59628176"
server_key_iv: "adbad7296be9b3c397d47b58fafcbdf1"
server_key_version: "encryptionkey@0"
I20260810 06:48:27.679632 8626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:27.732923 8626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:27.734308 8626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:27.734766 8626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:27.736302 8626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:27.738636 8626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:27.738787 8626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:27.738945 8626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:27.739053 8626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:27.789585 8626 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:45157
I20260810 06:48:27.789669 8856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:45157 every 8 connection(s)
I20260810 06:48:27.791712 8626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:48:27.799757 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 8626
I20260810 06:48:27.800083 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:48:27.800777 11208 external_mini_cluster.cc:1428] Setting key 23499d9df93dfef9e1c980407348ab5c
I20260810 06:48:27.805284 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46189
--builtin_ntp_servers=127.10.242.20:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:48:27.812103 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:27.795697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:48397 (local address 127.10.242.62:46189)
0810 06:48:27.796415 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:27.796427 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:27.798480 (+ 2053us) server_negotiation.cc:407] Connection header received
0810 06:48:27.801612 (+ 3132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:27.801628 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:27.801817 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:27.801982 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:27.803974 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:27.805092 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:27.807784 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:27.808450 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:27.808650 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:27.810728 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:27.810742 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:27.810745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:27.810774 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:27.810835 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:27.810850 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:27.810990 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:27.811310 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 06:48:27.811499 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":298,"threads_started":1}
I20260810 06:48:27.813845 8864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:27.796782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:46189 (local address 127.10.242.2:48397)
0810 06:48:27.798255 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:27.798323 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:27.801116 (+ 2793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:27.802463 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:27.802480 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:27.802641 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:27.803718 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:27.803756 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:27.805357 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:27.805365 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:27.807567 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:27.807595 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:27.808783 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:27.808792 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:27.809208 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:27.810224 (+ 1016us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:48:27.810228 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:27.810307 (+ 79us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:48:27.810446 (+ 139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:27.810983 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:27.811007 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:27.811018 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:48:27.811263 (+ 245us) client_negotiation.cc:240] Negotiation successful
0810 06:48:27.811997 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":383,"threads_started":1}
I20260810 06:48:27.816183 8859 heartbeater.cc:347] Connected to a master server at 127.10.242.62:46189
I20260810 06:48:27.816653 8859 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:27.817862 8859 heartbeater.cc:511] Master 127.10.242.62:46189 requested a full tablet report, sending...
I20260810 06:48:27.820286 8291 ts_manager.cc:194] Registered new tserver with Master: b8d17a646bae454fb4bb13ed236be943 (127.10.242.2:45157)
I20260810 06:48:27.822083 8291 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.2:48397
I20260810 06:48:28.094515 8620 heartbeater.cc:503] Master 127.10.242.62:46189 was elected leader, sending a full tablet report...
W20260810 06:48:28.352378 8866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:28.352880 8866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:28.353111 8866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:28.364454 8866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:28.364626 8866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:28.364863 8866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:48:28.379115 8866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:46189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:28.381238 8866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:28.384265 8866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:28.404402 8880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:28.404908 8884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:28.405316 8881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:28.406308 8866 server_base.cc:1034] running on GCE node
I20260810 06:48:28.407258 8866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:28.408681 8866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:28.409953 8866 hybrid_clock.cc:648] HybridClock initialized: now 1786344508409887 us; error 65 us; skew 500 ppm
I20260810 06:48:28.410487 8866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:28.413827 8866 webserver.cc:460] Webserver started at http://127.10.242.3:34525/ using document root <none> and password file <none>
I20260810 06:48:28.414934 8866 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:28.415098 8866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:28.415582 8866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:28.420188 8866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "bf374910fb9846fc9792acae182e93bc"
format_stamp: "Formatted at 2026-08-10 06:48:28 on dist-test-slave-zj8q"
server_key: "6b6658c3a8a939f0546eac1c858c0601"
server_key_iv: "1634b64a8f4fc4990a6c36e779528b83"
server_key_version: "encryptionkey@0"
I20260810 06:48:28.421485 8866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "bf374910fb9846fc9792acae182e93bc"
format_stamp: "Formatted at 2026-08-10 06:48:28 on dist-test-slave-zj8q"
server_key: "6b6658c3a8a939f0546eac1c858c0601"
server_key_iv: "1634b64a8f4fc4990a6c36e779528b83"
server_key_version: "encryptionkey@0"
I20260810 06:48:28.429760 8866 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260810 06:48:28.435159 8896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:28.437323 8866 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260810 06:48:28.437552 8866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "bf374910fb9846fc9792acae182e93bc"
format_stamp: "Formatted at 2026-08-10 06:48:28 on dist-test-slave-zj8q"
server_key: "6b6658c3a8a939f0546eac1c858c0601"
server_key_iv: "1634b64a8f4fc4990a6c36e779528b83"
server_key_version: "encryptionkey@0"
I20260810 06:48:28.437829 8866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:28.490978 8866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:28.492400 8866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:28.492857 8866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:28.494432 8866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:28.496845 8866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:28.497031 8866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:28.497213 8866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:28.497329 8866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:28.548802 8866 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:34643
I20260810 06:48:28.548872 9079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:34643 every 8 connection(s)
I20260810 06:48:28.550959 8866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:48:28.552664 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 8866
I20260810 06:48:28.552960 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:48:28.554064 11208 external_mini_cluster.cc:1428] Setting key 414c72e9828313da7e448636afa62c2b
I20260810 06:48:28.572544 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.555672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:58047 (local address 127.10.242.62:46189)
0810 06:48:28.556630 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.556640 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.559008 (+ 2368us) server_negotiation.cc:407] Connection header received
0810 06:48:28.561748 (+ 2740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.561760 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.561861 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.562008 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:28.563760 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.564928 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.567661 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.568327 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.568471 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.571108 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:28.571148 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:28.571155 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:28.571196 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:28.571264 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:28.571289 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:28.571419 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.571850 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.571972 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":547,"threads_started":1}
I20260810 06:48:28.574373 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.556954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:46189 (local address 127.10.242.3:58047)
0810 06:48:28.558775 (+ 1821us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:28.558876 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:28.561399 (+ 2523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:28.562332 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:28.562351 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:28.562509 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:28.563527 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.563563 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.565140 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.565146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.567473 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.567496 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.568707 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.568723 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.569136 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.570401 (+ 1265us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:48:28.570408 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:28.570508 (+ 100us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:48:28.570764 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:28.571457 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:28.571476 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:28.571498 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:48:28.571835 (+ 337us) client_negotiation.cc:240] Negotiation successful
0810 06:48:28.572531 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":426,"threads_started":1}
I20260810 06:48:28.576311 9080 heartbeater.cc:347] Connected to a master server at 127.10.242.62:46189
I20260810 06:48:28.576779 9080 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:28.577996 9080 heartbeater.cc:511] Master 127.10.242.62:46189 requested a full tablet report, sending...
I20260810 06:48:28.580287 8291 ts_manager.cc:194] Registered new tserver with Master: bf374910fb9846fc9792acae182e93bc (127.10.242.3:34643)
I20260810 06:48:28.581610 8291 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.3:58047
I20260810 06:48:28.586244 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:48:28.601491 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.592286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.10.242.62:46189)
0810 06:48:28.592511 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.592518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.593281 (+ 763us) server_negotiation.cc:407] Connection header received
0810 06:48:28.593522 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.593534 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.593623 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.593753 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:28.594535 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.595577 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.597804 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.598550 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.598778 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.600263 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:28.600276 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:28.600280 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:28.600307 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:28.600364 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:28.600388 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:28.600548 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.600797 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.600970 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:48:28.606757 8291 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:48:28.615217 8291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:48:28.641636 9106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.626776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:45157 (local address 127.0.0.1:60228)
0810 06:48:28.629147 (+ 2371us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:28.629165 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:28.629280 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:28.630476 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:28.630485 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:28.630519 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:28.632102 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.632117 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.635227 (+ 3110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.635233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.637719 (+ 2486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.637736 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.640636 (+ 2900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.640643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.640893 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.640949 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 06:48:28.641084 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 06:48:28.641272 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2197,"mutex_wait_us":22,"thread_start_us":2039,"threads_started":1}
I20260810 06:48:28.641692 9110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.627306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60228 (local address 127.10.242.2:45157)
0810 06:48:28.629414 (+ 2108us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.629424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.629456 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:48:28.629571 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.629580 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.629945 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.630168 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:28.632347 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.634022 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.638736 (+ 4714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.640165 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.640294 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.640470 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.641155 (+ 685us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.641339 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":946,"thread_start_us":829,"threads_started":1}
I20260810 06:48:28.645921 8743 tablet_service.cc:1467] Processing CreateTablet for tablet 06e3b81a98394153a8c4eebbda912f2a (DEFAULT_TABLE table=test-table [id=fde85c7c425e4ae28f9ab944db8db401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:28.646943 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.626609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:34643 (local address 127.0.0.1:43430)
0810 06:48:28.627600 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:28.627646 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:28.627847 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:28.639954 (+ 12107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:28.639959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:28.639983 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:28.640386 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.640399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.642305 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.642310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.644442 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.644458 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.646145 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.646175 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.646332 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.646364 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:48:28.646460 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:48:28.646586 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":39,"thread_start_us":508,"threads_started":1}
I20260810 06:48:28.646989 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.626881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.10.242.3:34643)
0810 06:48:28.639412 (+ 12531us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.639420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.639457 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:48:28.639554 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.639562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.639770 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.639954 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:28.640585 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.642104 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.644602 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.645933 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.646068 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.646363 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.646506 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.646653 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11367,"thread_start_us":11192,"threads_started":1}
I20260810 06:48:28.647495 8743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06e3b81a98394153a8c4eebbda912f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:28.647998 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.629323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45242 (local address 127.10.242.1:45947)
0810 06:48:28.638792 (+ 9469us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.638803 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.639439 (+ 636us) server_negotiation.cc:407] Connection header received
0810 06:48:28.639592 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.639599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.639832 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.640015 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:28.640868 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.642280 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.644870 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.646273 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.646400 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.646760 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.647524 (+ 764us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.647674 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9272,"thread_start_us":9124,"threads_started":1}
I20260810 06:48:28.647994 9111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.628833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.0.0.1:45242)
0810 06:48:28.639311 (+ 10478us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:28.639329 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:28.639453 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:28.640244 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:28.640253 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:28.640275 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:28.640706 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.640716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.642447 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.642456 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.644671 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.644687 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.646902 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.646909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.647354 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.647379 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:48:28.647461 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:48:28.647663 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10369,"mutex_wait_us":4,"thread_start_us":10201,"threads_started":1}
I20260810 06:48:28.650628 8971 tablet_service.cc:1467] Processing CreateTablet for tablet 06e3b81a98394153a8c4eebbda912f2a (DEFAULT_TABLE table=test-table [id=fde85c7c425e4ae28f9ab944db8db401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:28.652072 8971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06e3b81a98394153a8c4eebbda912f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:28.657356 8499 tablet_service.cc:1467] Processing CreateTablet for tablet 06e3b81a98394153a8c4eebbda912f2a (DEFAULT_TABLE table=test-table [id=fde85c7c425e4ae28f9ab944db8db401]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:28.658468 8499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06e3b81a98394153a8c4eebbda912f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:28.662338 9113 tablet_bootstrap.cc:492] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: Bootstrap starting.
I20260810 06:48:28.666057 9113 tablet_bootstrap.cc:654] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:28.666532 9116 tablet_bootstrap.cc:492] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: Bootstrap starting.
I20260810 06:48:28.667452 9113 log.cc:824] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:28.669632 9116 tablet_bootstrap.cc:654] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:28.670722 9113 tablet_bootstrap.cc:492] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: No bootstrap required, opened a new log
I20260810 06:48:28.670743 9116 log.cc:824] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:28.670998 9113 ts_tablet_manager.cc:1397] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260810 06:48:28.670975 9120 tablet_bootstrap.cc:492] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b: Bootstrap starting.
I20260810 06:48:28.673902 9116 tablet_bootstrap.cc:492] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: No bootstrap required, opened a new log
I20260810 06:48:28.674218 9116 ts_tablet_manager.cc:1397] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 06:48:28.674782 9120 tablet_bootstrap.cc:654] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:28.675995 9120 log.cc:824] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:28.678984 9120 tablet_bootstrap.cc:492] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b: No bootstrap required, opened a new log
I20260810 06:48:28.678789 9113 raft_consensus.cc:348] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.679114 9113 raft_consensus.cc:374] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:28.679206 9113 raft_consensus.cc:729] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d17a646bae454fb4bb13ed236be943, State: Initialized, Role: FOLLOWER
I20260810 06:48:28.679256 9120 ts_tablet_manager.cc:1397] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 06:48:28.679805 9113 consensus_queue.cc:260] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.680857 8859 heartbeater.cc:503] Master 127.10.242.62:46189 was elected leader, sending a full tablet report...
I20260810 06:48:28.681459 9113 ts_tablet_manager.cc:1428] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260810 06:48:28.684921 9116 raft_consensus.cc:348] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.685325 9116 raft_consensus.cc:374] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:28.685428 9116 raft_consensus.cc:729] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf374910fb9846fc9792acae182e93bc, State: Initialized, Role: FOLLOWER
I20260810 06:48:28.686048 9116 consensus_queue.cc:260] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.687131 9080 heartbeater.cc:503] Master 127.10.242.62:46189 was elected leader, sending a full tablet report...
I20260810 06:48:28.687814 9116 ts_tablet_manager.cc:1428] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260810 06:48:28.689204 9120 raft_consensus.cc:348] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.689591 9120 raft_consensus.cc:374] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:28.689687 9120 raft_consensus.cc:729] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a96b6d3a0414d748136f5edec95858b, State: Initialized, Role: FOLLOWER
I20260810 06:48:28.690250 9120 consensus_queue.cc:260] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.691648 9120 ts_tablet_manager.cc:1428] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260810 06:48:28.739251 9129 raft_consensus.cc:482] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:28.739539 9129 raft_consensus.cc:504] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.741117 9129 leader_election.cc:288] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf374910fb9846fc9792acae182e93bc (127.10.242.3:34643), b8d17a646bae454fb4bb13ed236be943 (127.10.242.2:45157)
I20260810 06:48:28.749747 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.741570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:53495 (local address 127.10.242.3:34643)
0810 06:48:28.741755 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.741761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.742626 (+ 865us) server_negotiation.cc:407] Connection header received
0810 06:48:28.742777 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.742782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.742851 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.742932 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:28.743600 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.744740 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.747593 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.748802 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.748972 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.749040 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.749383 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.749499 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:48:28.749841 9130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.741491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:34643 (local address 127.10.242.1:53495)
0810 06:48:28.742530 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:28.742550 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:28.742665 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:28.742943 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:28.742946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:28.742964 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:28.743444 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.743461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.744870 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.744875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.747392 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.747402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.748966 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.748971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.749141 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.749239 (+ 98us) client_negotiation.cc:769] Sending connection context
0810 06:48:28.749327 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:48:28.749452 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":195,"thread_start_us":420,"threads_started":1}
I20260810 06:48:28.749946 9110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.741697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:60809 (local address 127.10.242.2:45157)
0810 06:48:28.741878 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.741883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.742490 (+ 607us) server_negotiation.cc:407] Connection header received
0810 06:48:28.742705 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.742711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.742807 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.742907 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:28.743680 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.744740 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.747833 (+ 3093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.748961 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.749091 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.749138 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.749483 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.749603 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:48:28.750561 9132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.741348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:45157 (local address 127.10.242.1:60809)
0810 06:48:28.742375 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:28.742396 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:28.742547 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:28.742925 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:28.742930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:28.742957 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:28.743538 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.743549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.744866 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.744872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.747386 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.747402 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.749135 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.749141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.749346 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.749373 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 06:48:28.749443 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 06:48:28.749540 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":152,"thread_start_us":879,"threads_started":1}
I20260810 06:48:28.750790 9127 raft_consensus.cc:482] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:28.751075 9127 raft_consensus.cc:504] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.751433 8776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e3b81a98394153a8c4eebbda912f2a" candidate_uuid: "7a96b6d3a0414d748136f5edec95858b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d17a646bae454fb4bb13ed236be943" is_pre_election: true
I20260810 06:48:28.751520 9013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e3b81a98394153a8c4eebbda912f2a" candidate_uuid: "7a96b6d3a0414d748136f5edec95858b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf374910fb9846fc9792acae182e93bc" is_pre_election: true
I20260810 06:48:28.751936 8776 raft_consensus.cc:2454] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a96b6d3a0414d748136f5edec95858b in term 0.
I20260810 06:48:28.751940 9013 raft_consensus.cc:2454] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a96b6d3a0414d748136f5edec95858b in term 0.
I20260810 06:48:28.752616 8423 leader_election.cc:302] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a96b6d3a0414d748136f5edec95858b, b8d17a646bae454fb4bb13ed236be943; no voters:
I20260810 06:48:28.752974 9129 raft_consensus.cc:2781] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:48:28.752974 9127 leader_election.cc:288] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d17a646bae454fb4bb13ed236be943 (127.10.242.2:45157), 7a96b6d3a0414d748136f5edec95858b (127.10.242.1:45947)
I20260810 06:48:28.753129 9129 raft_consensus.cc:482] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:48:28.753228 9129 raft_consensus.cc:3037] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:28.755820 9129 raft_consensus.cc:504] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.756390 9129 leader_election.cc:288] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [CANDIDATE]: Term 1 election: Requested vote from peers bf374910fb9846fc9792acae182e93bc (127.10.242.3:34643), b8d17a646bae454fb4bb13ed236be943 (127.10.242.2:45157)
I20260810 06:48:28.756788 9013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e3b81a98394153a8c4eebbda912f2a" candidate_uuid: "7a96b6d3a0414d748136f5edec95858b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf374910fb9846fc9792acae182e93bc"
I20260810 06:48:28.756996 9013 raft_consensus.cc:3037] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:28.757138 8776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e3b81a98394153a8c4eebbda912f2a" candidate_uuid: "7a96b6d3a0414d748136f5edec95858b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d17a646bae454fb4bb13ed236be943"
I20260810 06:48:28.757326 8776 raft_consensus.cc:3037] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:28.760363 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.753345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:45157 (local address 127.10.242.3:53515)
0810 06:48:28.753573 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:28.753587 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:28.753663 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:28.754841 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:28.754845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:28.754877 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:28.755010 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.755017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.755960 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.755965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.757766 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.757781 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.759570 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.759576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.759812 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.759845 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:48:28.759927 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:48:28.760056 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260810 06:48:28.760548 8776 raft_consensus.cc:2454] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a96b6d3a0414d748136f5edec95858b in term 1.
I20260810 06:48:28.760558 9013 raft_consensus.cc:2454] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a96b6d3a0414d748136f5edec95858b in term 1.
I20260810 06:48:28.760950 8423 leader_election.cc:302] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a96b6d3a0414d748136f5edec95858b, b8d17a646bae454fb4bb13ed236be943; no voters:
I20260810 06:48:28.761013 9110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.754419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:53515 (local address 127.10.242.2:45157)
0810 06:48:28.754583 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.754589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.754605 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:48:28.754647 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.754651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.754713 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.754787 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:28.755135 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.755840 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.757945 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.759266 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.759414 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.759479 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.760646 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.760780 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:48:28.761346 9129 raft_consensus.cc:2781] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:28.761654 8775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e3b81a98394153a8c4eebbda912f2a" candidate_uuid: "bf374910fb9846fc9792acae182e93bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d17a646bae454fb4bb13ed236be943" is_pre_election: true
I20260810 06:48:28.762068 8775 raft_consensus.cc:2379] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf374910fb9846fc9792acae182e93bc in current term 1: Already voted for candidate 7a96b6d3a0414d748136f5edec95858b in this term.
I20260810 06:48:28.762933 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.754635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:32937 (local address 127.10.242.1:45947)
0810 06:48:28.756614 (+ 1979us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.756619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.756636 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:48:28.756690 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.756694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.756773 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.756864 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:28.757573 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.758566 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.761055 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.762043 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.762139 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.762224 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.762603 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.762723 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1881}
I20260810 06:48:28.763433 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.753352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.10.242.3:32937)
0810 06:48:28.754026 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:28.754043 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:28.754119 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:28.756898 (+ 2779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:28.756903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:28.756928 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:28.757369 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.757381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.758761 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.758766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.760822 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:28.760836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.762336 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.762340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:28.762449 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.762467 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 06:48:28.763083 (+ 616us) client_negotiation.cc:240] Negotiation successful
0810 06:48:28.763213 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":34,"thread_start_us":377,"threads_started":1}
I20260810 06:48:28.764245 8534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e3b81a98394153a8c4eebbda912f2a" candidate_uuid: "bf374910fb9846fc9792acae182e93bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a96b6d3a0414d748136f5edec95858b" is_pre_election: true
I20260810 06:48:28.766358 9129 raft_consensus.cc:686] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 1 LEADER]: Becoming Leader. State: Replica: 7a96b6d3a0414d748136f5edec95858b, State: Running, Role: LEADER
I20260810 06:48:28.766834 9129 consensus_queue.cc:237] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } }
I20260810 06:48:28.770592 8902 leader_election.cc:302] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf374910fb9846fc9792acae182e93bc; no voters: 7a96b6d3a0414d748136f5edec95858b, b8d17a646bae454fb4bb13ed236be943
I20260810 06:48:28.770931 9127 raft_consensus.cc:2726] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 06:48:28.771286 8291 catalog_manager.cc:5495] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a96b6d3a0414d748136f5edec95858b (127.10.242.1). New cstate: current_term: 1 leader_uuid: "7a96b6d3a0414d748136f5edec95858b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a96b6d3a0414d748136f5edec95858b" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 45947 } health_report { overall_health: HEALTHY } } }
I20260810 06:48:28.788136 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.783095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.10.242.1:45947)
0810 06:48:28.783269 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.783275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.783332 (+ 57us) server_negotiation.cc:407] Connection header received
0810 06:48:28.783530 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.783534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.783655 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.783772 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:48:28.784063 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.784817 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.786251 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.786677 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.786807 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.787208 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:48:28.787689 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:48:28.787797 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.787834 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.787891 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 06:48:28.794848 8860 tablet.cc:2369] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:28.795874 8775 raft_consensus.cc:1261] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 1 FOLLOWER]: Refusing update from remote peer 7a96b6d3a0414d748136f5edec95858b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:28.795886 9013 raft_consensus.cc:1261] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 1 FOLLOWER]: Refusing update from remote peer 7a96b6d3a0414d748136f5edec95858b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:28.796802 9138 consensus_queue.cc:1035] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf374910fb9846fc9792acae182e93bc" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:28.797125 9129 consensus_queue.cc:1035] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d17a646bae454fb4bb13ed236be943" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 45157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:48:28.803562 9081 tablet.cc:2369] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:28.824447 8291 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:28.824695 8291 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:28.828269 8291 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49e95663cdbc4c1c9487d8f174d1c000: Sending DeleteTablet for 3 replicas of tablet 06e3b81a98394153a8c4eebbda912f2a
I20260810 06:48:28.829383 8499 tablet_service.cc:1514] Processing DeleteTablet for tablet 06e3b81a98394153a8c4eebbda912f2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:28 UTC) from {username='slave'} at 127.0.0.1:45242
I20260810 06:48:28.829366 8971 tablet_service.cc:1514] Processing DeleteTablet for tablet 06e3b81a98394153a8c4eebbda912f2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:28 UTC) from {username='slave'} at 127.0.0.1:43430
I20260810 06:48:28.829525 8743 tablet_service.cc:1514] Processing DeleteTablet for tablet 06e3b81a98394153a8c4eebbda912f2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:28 UTC) from {username='slave'} at 127.0.0.1:60228
I20260810 06:48:28.831664 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 8397
I20260810 06:48:28.839282 9169 tablet_replica.cc:330] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: stopping tablet replica
I20260810 06:48:28.839732 9169 raft_consensus.cc:2229] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:28.840235 9169 raft_consensus.cc:2258] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:28.842257 9166 tablet_replica.cc:330] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: stopping tablet replica
I20260810 06:48:28.842355 9169 ts_tablet_manager.cc:1905] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:28.842836 9166 raft_consensus.cc:2229] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:28.843348 9166 raft_consensus.cc:2258] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:28.845423 9166 ts_tablet_manager.cc:1905] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:28.849300 9169 ts_tablet_manager.cc:1918] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:28.849525 9169 log.cc:1196] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/06e3b81a98394153a8c4eebbda912f2a
I20260810 06:48:28.850128 9169 ts_tablet_manager.cc:1939] T 06e3b81a98394153a8c4eebbda912f2a P b8d17a646bae454fb4bb13ed236be943: Deleting consensus metadata
I20260810 06:48:28.852043 8265 catalog_manager.cc:4841] TS b8d17a646bae454fb4bb13ed236be943 (127.10.242.2:45157): tablet 06e3b81a98394153a8c4eebbda912f2a (table test-table [id=fde85c7c425e4ae28f9ab944db8db401]) successfully deleted
I20260810 06:48:28.852720 9166 ts_tablet_manager.cc:1918] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:28.852931 9166 log.cc:1196] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/06e3b81a98394153a8c4eebbda912f2a
I20260810 06:48:28.853492 9166 ts_tablet_manager.cc:1939] T 06e3b81a98394153a8c4eebbda912f2a P bf374910fb9846fc9792acae182e93bc: Deleting consensus metadata
I20260810 06:48:28.855402 9167 tablet_replica.cc:330] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b: stopping tablet replica
I20260810 06:48:28.855405 8264 catalog_manager.cc:4841] TS bf374910fb9846fc9792acae182e93bc (127.10.242.3:34643): tablet 06e3b81a98394153a8c4eebbda912f2a (table test-table [id=fde85c7c425e4ae28f9ab944db8db401]) successfully deleted
I20260810 06:48:28.855834 9167 raft_consensus.cc:2229] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:48:28.856510 9167 raft_consensus.cc:2258] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:28.858554 9167 ts_tablet_manager.cc:1905] T 06e3b81a98394153a8c4eebbda912f2a P 7a96b6d3a0414d748136f5edec95858b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:28.861447 9112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.833154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45270 (local address 127.10.242.1:45947)
0810 06:48:28.833341 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.833348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.854695 (+ 21347us) server_negotiation.cc:407] Connection header received
0810 06:48:28.854907 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.854912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.855069 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.855192 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:28.856070 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.856997 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.858323 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.858811 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.858932 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.860022 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:28.860030 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:28.860047 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:28.860679 (+ 632us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:28.860726 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:28.860746 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:28.860866 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:28.860994 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 06:48:28.861099 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:48:28.862114 8612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:28.961011 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 8397
W20260810 06:48:28.992597 8262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.1:45947 (error 108)
I20260810 06:48:28.992854 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 8626
I20260810 06:48:28.994820 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.994307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.0.0.1:45272)
0810 06:48:28.994498 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:28.994628 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 06:48:28.995178 8262 catalog_manager.cc:4584] TS 7a96b6d3a0414d748136f5edec95858b (127.10.242.1:45947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06e3b81a98394153a8c4eebbda912f2a: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45947: connect: Connection refused (error 111)
I20260810 06:48:29.001331 9110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:28.993767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60232 (local address 127.10.242.2:45157)
0810 06:48:28.993982 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:28.993988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:28.994639 (+ 651us) server_negotiation.cc:407] Connection header received
0810 06:48:28.994818 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:28.994824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:28.994934 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:28.995028 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:28.995524 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.996264 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.998403 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:28.998795 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:28.998894 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:28.999986 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:28.999994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:29.000009 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:29.000582 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:29.000635 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:29.000663 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:29.000766 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:29.000901 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:48:29.000979 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:48:29.002029 8849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:29.048836 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:29.048359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.0.0.1:45276)
0810 06:48:29.048561 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:29.048657 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 06:48:29.049855 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:29.049505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.0.0.1:45280)
0810 06:48:29.049653 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:29.049725 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 06:48:29.110809 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:29.110229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.0.0.1:45286)
0810 06:48:29.110464 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:29.110598 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 06:48:29.112098 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:29.111672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.0.0.1:45294)
0810 06:48:29.111844 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:29.111937 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 06:48:29.119190 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 8626
I20260810 06:48:29.156886 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 8866
I20260810 06:48:29.163906 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:29.157671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.10.242.3:34643)
0810 06:48:29.157864 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:29.157870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:29.157891 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:48:29.158071 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:29.158077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:29.158208 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:29.158300 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:29.158749 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:29.159476 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:29.160799 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:29.161210 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:29.161318 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:29.162549 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:29.162559 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:29.162576 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:29.163176 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:29.163223 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:29.163243 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:29.163343 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:29.163480 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 06:48:29.163593 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:48:29.164490 9074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:29.192732 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:29.192112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.0.0.1:45310)
0810 06:48:29.192368 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:29.192530 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 06:48:29.194027 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:29.193510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.0.0.1:45318)
0810 06:48:29.193766 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:29.193862 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260810 06:48:29.254969 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 8866
I20260810 06:48:29.275904 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 8238
I20260810 06:48:29.276997 8352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:29.383010 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:29.382391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:45947 (local address 127.0.0.1:45324)
0810 06:48:29.382587 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:29.382709 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:45947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:48:29.383265 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 8238
2026-08-10T06:48:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3354 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:29 dist-test-slave-zj8q krb5kdc[9187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:29 dist-test-slave-zj8q krb5kdc[9187](info): set up 2 sockets
Aug 10 06:48:29 dist-test-slave-zj8q krb5kdc[9187](info): commencing operation
krb5kdc: starting...
W20260810 06:48:30.648471 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:30 dist-test-slave-zj8q krb5kdc[9187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:48:30.930861 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:32867
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:32867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:31.260731 9212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:31.260982 9212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:31.261111 9212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:31.267976 9212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:31.268067 9212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:31.268141 9212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:31.268224 9212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:31.268282 9212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:31.277575 9212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:32867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:32867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:31.278760 9212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:31.280037 9212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:31.291328 9223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:31.291352 9220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:31.291644 9219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:31.292028 9212 server_base.cc:1034] running on GCE node
I20260810 06:48:31.292781 9212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:31.294067 9212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:31.295269 9212 hybrid_clock.cc:648] HybridClock initialized: now 1786344511295228 us; error 31 us; skew 500 ppm
I20260810 06:48:31.295583 9212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:31 dist-test-slave-zj8q krb5kdc[9187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344511, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:31.298548 9212 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:48:31.299888 9212 webserver.cc:460] Webserver started at http://127.10.242.62:34473/ using document root <none> and password file <none>
I20260810 06:48:31.300515 9212 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:31.300630 9212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:31.300908 9212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:31.303844 9212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "005ae545f7be407da9e82a101d9d8b24"
format_stamp: "Formatted at 2026-08-10 06:48:31 on dist-test-slave-zj8q"
server_key: "a0b47d811714658669e324dd7e75a7a8"
server_key_iv: "a1a3c02515eb1bef4554c4e463a1c481"
server_key_version: "encryptionkey@0"
I20260810 06:48:31.304560 9212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "005ae545f7be407da9e82a101d9d8b24"
format_stamp: "Formatted at 2026-08-10 06:48:31 on dist-test-slave-zj8q"
server_key: "a0b47d811714658669e324dd7e75a7a8"
server_key_iv: "a1a3c02515eb1bef4554c4e463a1c481"
server_key_version: "encryptionkey@0"
I20260810 06:48:31.309466 9212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:48:31.312779 9234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:31.313983 9212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:48:31.314131 9212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "005ae545f7be407da9e82a101d9d8b24"
format_stamp: "Formatted at 2026-08-10 06:48:31 on dist-test-slave-zj8q"
server_key: "a0b47d811714658669e324dd7e75a7a8"
server_key_iv: "a1a3c02515eb1bef4554c4e463a1c481"
server_key_version: "encryptionkey@0"
I20260810 06:48:31.314347 9212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:31.326251 9212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:31.329824 9212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:31.330073 9212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:31.346939 9212 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:32867
I20260810 06:48:31.346961 9311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:32867 every 8 connection(s)
I20260810 06:48:31.348188 9212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:31.351661 9312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:31.354904 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 9212
I20260810 06:48:31.355072 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:31.355518 11208 external_mini_cluster.cc:1428] Setting key 8a9e57ab3d3e4fac43c90ef7545f8d82
I20260810 06:48:31.363441 9312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24: Bootstrap starting.
I20260810 06:48:31.367018 9312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:31.368247 9312 log.cc:824] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:48:31 dist-test-slave-zj8q krb5kdc[9187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:31.371258 9312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24: No bootstrap required, opened a new log
I20260810 06:48:31.377803 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:31.357205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.10.242.62:32867)
0810 06:48:31.357914 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:31.357925 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:31.357965 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:48:31.359605 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:31.359628 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:31.359991 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:31.360298 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:31.362079 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:31.363384 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:31.365202 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:31.365846 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:31.365962 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:31.370914 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:31.370921 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:31.370935 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:31.370968 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:31.373909 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:31.374740 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:31.374747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:31.374753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:31.374936 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:31.375330 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:31.375333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:31.375335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:31.375833 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:31.376058 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:31.376466 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 06:48:31.376779 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":320,"threads_started":1}
I20260810 06:48:31.379719 9312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005ae545f7be407da9e82a101d9d8b24" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 32867 } }
I20260810 06:48:31.380055 9312 raft_consensus.cc:374] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:31.380154 9312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005ae545f7be407da9e82a101d9d8b24, State: Initialized, Role: FOLLOWER
I20260810 06:48:31.380722 9312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005ae545f7be407da9e82a101d9d8b24" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 32867 } }
I20260810 06:48:31.380944 9312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:31.381071 9312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:31.381212 9312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:31.383370 9312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005ae545f7be407da9e82a101d9d8b24" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 32867 } }
I20260810 06:48:31.383824 9312 leader_election.cc:302] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 005ae545f7be407da9e82a101d9d8b24; no voters:
I20260810 06:48:31.384334 9312 leader_election.cc:288] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:31.384434 9322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:31.385124 9322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [term 1 LEADER]: Becoming Leader. State: Replica: 005ae545f7be407da9e82a101d9d8b24, State: Running, Role: LEADER
I20260810 06:48:31.385682 9322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005ae545f7be407da9e82a101d9d8b24" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 32867 } }
I20260810 06:48:31.386296 9312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:31.388415 9326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 005ae545f7be407da9e82a101d9d8b24. Latest consensus state: current_term: 1 leader_uuid: "005ae545f7be407da9e82a101d9d8b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005ae545f7be407da9e82a101d9d8b24" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 32867 } } }
I20260810 06:48:31.388391 9324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "005ae545f7be407da9e82a101d9d8b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005ae545f7be407da9e82a101d9d8b24" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 32867 } } }
I20260810 06:48:31.388943 9324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:31.388943 9326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:31.391002 9335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:31.395282 9335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:31.402767 9335 catalog_manager.cc:1347] Generated new cluster ID: 816d9d1e5d034f4f8e9ece92ac101440
I20260810 06:48:31.402869 9335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:31.431501 9335 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:31.433334 9335 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:31.440758 9335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24: Generated new TSK 0
I20260810 06:48:31.441695 9335 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:48:31.517012 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:32867
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:31.846789 9358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:31.847085 9358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:31.847244 9358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:31.854319 9358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:31.854435 9358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:31.854601 9358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:48:31.864495 9358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:32867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:31.865779 9358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:31.867144 9358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:31.879724 9374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:31.879793 9372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:31.879769 9371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:31.881080 9358 server_base.cc:1034] running on GCE node
I20260810 06:48:31.881614 9358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:31.882573 9358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:31.883770 9358 hybrid_clock.cc:648] HybridClock initialized: now 1786344511883725 us; error 38 us; skew 500 ppm
I20260810 06:48:31.884069 9358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:31 dist-test-slave-zj8q krb5kdc[9187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344511, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:31.887077 9358 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:48:31.888435 9358 webserver.cc:460] Webserver started at http://127.10.242.1:35333/ using document root <none> and password file <none>
I20260810 06:48:31.889055 9358 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:31.889180 9358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:31.889477 9358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:31.892448 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "ac937c431691477583feac80bb714cff"
format_stamp: "Formatted at 2026-08-10 06:48:31 on dist-test-slave-zj8q"
server_key: "1618735d46a955a24b87b9c9c12cf790"
server_key_iv: "aecb2380778a50ac63138cdc5a8119f9"
server_key_version: "encryptionkey@0"
I20260810 06:48:31.893205 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "ac937c431691477583feac80bb714cff"
format_stamp: "Formatted at 2026-08-10 06:48:31 on dist-test-slave-zj8q"
server_key: "1618735d46a955a24b87b9c9c12cf790"
server_key_iv: "aecb2380778a50ac63138cdc5a8119f9"
server_key_version: "encryptionkey@0"
I20260810 06:48:31.898036 9358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 06:48:31.901212 9383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:31.902442 9358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:48:31.902583 9358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "ac937c431691477583feac80bb714cff"
format_stamp: "Formatted at 2026-08-10 06:48:31 on dist-test-slave-zj8q"
server_key: "1618735d46a955a24b87b9c9c12cf790"
server_key_iv: "aecb2380778a50ac63138cdc5a8119f9"
server_key_version: "encryptionkey@0"
I20260810 06:48:31.902758 9358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:31.928519 9358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:31.932211 9358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:31.932461 9358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:31.933357 9358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:31.934629 9358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:31.934702 9358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:31.934818 9358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:31.934871 9358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:31.972546 9358 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:41775
I20260810 06:48:31.972580 9588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:41775 every 8 connection(s)
I20260810 06:48:31.973809 9358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:31.976837 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 9358
I20260810 06:48:31.976997 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:48:31.977633 11208 external_mini_cluster.cc:1428] Setting key 3c3259776c837f8861ad93e3eb06ddba
Aug 10 06:48:31 dist-test-slave-zj8q krb5kdc[9187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344511, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:31.995893 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:31.976273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:56985 (local address 127.10.242.62:32867)
0810 06:48:31.976752 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:31.976759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:31.978212 (+ 1453us) server_negotiation.cc:407] Connection header received
0810 06:48:31.980099 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:31.980103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:31.980166 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:31.980266 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:31.982202 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:31.982922 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:31.984922 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:31.985307 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:31.985397 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:31.989293 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:31.989298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:31.989303 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:31.989341 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:31.992715 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:31.993557 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:31.993564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:31.993568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:31.993731 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:31.994284 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:31.994288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:31.994290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:31.994557 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:31.994670 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:31.995470 (+ 800us) server_negotiation.cc:299] Negotiation successful
0810 06:48:31.995635 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":231,"threads_started":1}
I20260810 06:48:31.997162 9597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:31.976791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:32867 (local address 127.10.242.1:56985)
0810 06:48:31.977612 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:31.977685 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:31.979800 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:31.980532 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:31.980545 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:31.981223 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:31.982001 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:31.982026 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:31.983078 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:31.983083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:31.984796 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:31.984818 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:31.985533 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:31.985542 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:31.985815 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:31.986524 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:31.986550 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:31.989071 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:31.992840 (+ 3769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:31.992849 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:31.992864 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:31.993431 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:31.993886 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:31.993891 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:31.993895 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:31.994180 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:31.994678 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:31.994686 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:31.995081 (+ 395us) client_negotiation.cc:769] Sending connection context
0810 06:48:31.995392 (+ 311us) client_negotiation.cc:240] Negotiation successful
0810 06:48:31.995867 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":318,"threads_started":1}
I20260810 06:48:31.998667 9590 heartbeater.cc:347] Connected to a master server at 127.10.242.62:32867
I20260810 06:48:31.999008 9590 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:31.999800 9590 heartbeater.cc:511] Master 127.10.242.62:32867 requested a full tablet report, sending...
I20260810 06:48:32.002252 9260 ts_manager.cc:194] Registered new tserver with Master: ac937c431691477583feac80bb714cff (127.10.242.1:41775)
I20260810 06:48:32.004156 9260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:56985
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:48:32.047226 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:32867
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:32.383816 9603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:32.384073 9603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:32.384204 9603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:32.391372 9603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:32.391487 9603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:32.391649 9603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:48:32.401119 9603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:32867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:32.402349 9603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:32.403669 9603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:32.416337 9613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:32.416312 9616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:32.416574 9619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:32.417006 9603 server_base.cc:1034] running on GCE node
I20260810 06:48:32.417654 9603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:32.418638 9603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:32.419842 9603 hybrid_clock.cc:648] HybridClock initialized: now 1786344512419792 us; error 41 us; skew 500 ppm
I20260810 06:48:32.420135 9603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:32 dist-test-slave-zj8q krb5kdc[9187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344512, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:32.423076 9603 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:48:32.424465 9603 webserver.cc:460] Webserver started at http://127.10.242.2:40203/ using document root <none> and password file <none>
I20260810 06:48:32.425084 9603 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:32.425207 9603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:32.425503 9603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:32.428359 9603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "b70c5dc1f63c490d915c7632a8746c4d"
format_stamp: "Formatted at 2026-08-10 06:48:32 on dist-test-slave-zj8q"
server_key: "0cbfea1a5001863946028d35059fb842"
server_key_iv: "8397e17a3c1b08311a4ea2969804a7de"
server_key_version: "encryptionkey@0"
I20260810 06:48:32.429077 9603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "b70c5dc1f63c490d915c7632a8746c4d"
format_stamp: "Formatted at 2026-08-10 06:48:32 on dist-test-slave-zj8q"
server_key: "0cbfea1a5001863946028d35059fb842"
server_key_iv: "8397e17a3c1b08311a4ea2969804a7de"
server_key_version: "encryptionkey@0"
I20260810 06:48:32.434033 9603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260810 06:48:32.437258 9632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:32.438557 9603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:48:32.438699 9603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "b70c5dc1f63c490d915c7632a8746c4d"
format_stamp: "Formatted at 2026-08-10 06:48:32 on dist-test-slave-zj8q"
server_key: "0cbfea1a5001863946028d35059fb842"
server_key_iv: "8397e17a3c1b08311a4ea2969804a7de"
server_key_version: "encryptionkey@0"
I20260810 06:48:32.438871 9603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:32.457096 9603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:32.460662 9603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:32.460927 9603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:32.461844 9603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:32.463157 9603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:32.463243 9603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:32.463363 9603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:32.463418 9603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:32.492892 9603 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:43961
I20260810 06:48:32.492933 9820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:43961 every 8 connection(s)
I20260810 06:48:32.494086 9603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:48:32.495272 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 9603
I20260810 06:48:32.495462 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:48:32.496029 11208 external_mini_cluster.cc:1428] Setting key 2695c0307a2bac136c28a71f2fb59268
Aug 10 06:48:32 dist-test-slave-zj8q krb5kdc[9187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344512, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:32.513514 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:32.496776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:33181 (local address 127.10.242.62:32867)
0810 06:48:32.497238 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:32.497244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:32.498275 (+ 1031us) server_negotiation.cc:407] Connection header received
0810 06:48:32.500043 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:32.500049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:32.500124 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:32.500216 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:32.501696 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:32.502447 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:32.503961 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:32.504380 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:32.504485 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:32.507901 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:32.507906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:32.507909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:32.507940 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:32.510685 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:32.511349 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:32.511355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:32.511358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:32.511520 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:32.511884 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:32.511888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:32.511891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:32.512224 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:32.512358 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:32.512927 (+ 569us) server_negotiation.cc:299] Negotiation successful
0810 06:48:32.513141 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":1303,"threads_started":1}
I20260810 06:48:32.514214 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:32.497281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:32867 (local address 127.10.242.2:33181)
0810 06:48:32.498081 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:32.498174 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:32.499793 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:32.500457 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:32.500465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:32.500949 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:32.501531 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:32.501550 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:32.502595 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:32.502599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:32.503821 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:32.503835 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:32.504599 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:32.504608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:32.504824 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:32.505412 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:32.505441 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:32.507696 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:32.510813 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:32.510820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:32.510830 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:32.511236 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:32.511614 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:32.511617 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:32.511619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:32.511789 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:32.512330 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:32.512335 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:32.512612 (+ 277us) client_negotiation.cc:769] Sending connection context
0810 06:48:32.512822 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 06:48:32.513161 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":291,"threads_started":1}
I20260810 06:48:32.515564 9822 heartbeater.cc:347] Connected to a master server at 127.10.242.62:32867
I20260810 06:48:32.515861 9822 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:32.516731 9822 heartbeater.cc:511] Master 127.10.242.62:32867 requested a full tablet report, sending...
I20260810 06:48:32.518199 9260 ts_manager.cc:194] Registered new tserver with Master: b70c5dc1f63c490d915c7632a8746c4d (127.10.242.2:43961)
I20260810 06:48:32.519073 9260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:33181
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:48:32.558444 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:32867
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:32.885334 9836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:32.885599 9836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:32.885691 9836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:32.892815 9836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:32.892916 9836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:32.893044 9836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:48:32.902336 9836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:32867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:32.903461 9836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:32.904702 9836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:32.917395 9843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:32.917472 9850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:32.917362 9844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:32.918176 9836 server_base.cc:1034] running on GCE node
I20260810 06:48:32.918742 9836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:32.919591 9836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:32.920790 9836 hybrid_clock.cc:648] HybridClock initialized: now 1786344512920745 us; error 39 us; skew 500 ppm
I20260810 06:48:32.921073 9836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:32 dist-test-slave-zj8q krb5kdc[9187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344512, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:32.924050 9836 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:48:32.925369 9836 webserver.cc:460] Webserver started at http://127.10.242.3:33493/ using document root <none> and password file <none>
I20260810 06:48:32.925998 9836 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:32.926115 9836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:32.926437 9836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:32.929147 9836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "b3a041c8d90345d581ab73675ed36215"
format_stamp: "Formatted at 2026-08-10 06:48:32 on dist-test-slave-zj8q"
server_key: "0866a9304f4aede6c864e37d42e5919f"
server_key_iv: "ae0f45a2a52148832c5dbe3f3ef14744"
server_key_version: "encryptionkey@0"
I20260810 06:48:32.929896 9836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "b3a041c8d90345d581ab73675ed36215"
format_stamp: "Formatted at 2026-08-10 06:48:32 on dist-test-slave-zj8q"
server_key: "0866a9304f4aede6c864e37d42e5919f"
server_key_iv: "ae0f45a2a52148832c5dbe3f3ef14744"
server_key_version: "encryptionkey@0"
I20260810 06:48:32.934829 9836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 06:48:32.938107 9860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:32.939368 9836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:48:32.939517 9836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "b3a041c8d90345d581ab73675ed36215"
format_stamp: "Formatted at 2026-08-10 06:48:32 on dist-test-slave-zj8q"
server_key: "0866a9304f4aede6c864e37d42e5919f"
server_key_iv: "ae0f45a2a52148832c5dbe3f3ef14744"
server_key_version: "encryptionkey@0"
I20260810 06:48:32.939704 9836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:32.965497 9836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:32.969238 9836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:32.969497 9836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:32.970417 9836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:32.971706 9836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:32.971799 9836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:32.971917 9836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:32.971971 9836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:33.002597 9836 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:42803
I20260810 06:48:33.002684 10060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:42803 every 8 connection(s)
I20260810 06:48:33.003845 9836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:48:33.004787 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 9836
I20260810 06:48:33.004977 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:48:33.005511 11208 external_mini_cluster.cc:1428] Setting key 224c831a6560c7cce24ec95768cfbbb5
I20260810 06:48:33.007680 9590 heartbeater.cc:503] Master 127.10.242.62:32867 was elected leader, sending a full tablet report...
Aug 10 06:48:33 dist-test-slave-zj8q krb5kdc[9187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344512, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:33.024025 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.006740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:45867 (local address 127.10.242.62:32867)
0810 06:48:33.006932 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.006938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.008222 (+ 1284us) server_negotiation.cc:407] Connection header received
0810 06:48:33.010886 (+ 2664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.010890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.010953 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.011030 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:33.012399 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.013249 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.014821 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.015233 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.015346 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.018587 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:33.018592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:33.018596 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:33.018636 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:33.021594 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.022269 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.022272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.022274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.022397 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.022736 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.022739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.022741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.022988 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:33.023101 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.023631 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.023789 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:48:33.024878 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.007220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:32867 (local address 127.10.242.3:45867)
0810 06:48:33.008056 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:33.008108 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:33.010654 (+ 2546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:33.011221 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:33.011229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:33.011707 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:33.012248 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.012265 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.013381 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.013385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.014668 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.014685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.015444 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.015451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.015663 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.016225 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:33.016246 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:33.018398 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:33.021707 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:33.021713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:33.021723 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:33.022142 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:33.022475 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:33.022478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:33.022481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:33.022655 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:33.023087 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:33.023092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:33.023357 (+ 265us) client_negotiation.cc:769] Sending connection context
0810 06:48:33.023569 (+ 212us) client_negotiation.cc:240] Negotiation successful
0810 06:48:33.023911 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":367,"threads_started":1}
I20260810 06:48:33.026034 10062 heartbeater.cc:347] Connected to a master server at 127.10.242.62:32867
I20260810 06:48:33.026337 10062 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:33.026965 10062 heartbeater.cc:511] Master 127.10.242.62:32867 requested a full tablet report, sending...
I20260810 06:48:33.028250 9259 ts_manager.cc:194] Registered new tserver with Master: b3a041c8d90345d581ab73675ed36215 (127.10.242.3:42803)
I20260810 06:48:33.029047 9259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:45867
I20260810 06:48:33.032811 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:48:33.048332 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.036582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.10.242.62:32867)
0810 06:48:33.036757 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.036762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.037185 (+ 423us) server_negotiation.cc:407] Connection header received
0810 06:48:33.037314 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.037317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.037389 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.037459 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:33.038865 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.039553 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.040741 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.041144 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.041226 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.043824 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:33.043831 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:33.043834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:33.043864 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:33.046054 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.046761 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.046764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.046767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.046891 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.047234 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.047237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.047239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.047504 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:33.047722 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.047974 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.048106 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:48:33.051858 9259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:48:33.058522 9259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:48:33.080976 10079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.069648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:43961 (local address 127.0.0.1:55434)
0810 06:48:33.070583 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:33.070642 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:33.070821 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:33.072045 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:33.072054 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:33.072089 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:33.072578 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.072598 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.074895 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.074901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.077880 (+ 2979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.077898 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.079969 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.079976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.080200 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.080257 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 06:48:33.080365 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 06:48:33.080559 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":506,"threads_started":1}
I20260810 06:48:33.082665 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.069889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55434 (local address 127.10.242.2:43961)
0810 06:48:33.071380 (+ 1491us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.071390 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.071422 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:48:33.071535 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.071542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.071788 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.072832 (+ 1044us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:33.073055 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.074725 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.078110 (+ 3385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.079708 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.081791 (+ 2083us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.082026 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.082101 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.082299 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1353,"thread_start_us":1226,"threads_started":1}
I20260810 06:48:33.087147 9724 tablet_service.cc:1467] Processing CreateTablet for tablet f7b2e47538504286bac0082325cf3ec4 (DEFAULT_TABLE table=test-table [id=6c1e500db5df4fdd84b3544d0ee84753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:33.088507 9724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b2e47538504286bac0082325cf3ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:33.090519 10079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.070249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42803 (local address 127.0.0.1:47468)
0810 06:48:33.081234 (+ 10985us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:33.081258 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:33.081369 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:33.083825 (+ 2456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:33.083829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:33.083853 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:33.084009 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.084021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.085763 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.085768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.087630 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.087647 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.089749 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.089755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.089950 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.089982 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:48:33.090078 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:48:33.090237 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10860,"thread_start_us":15749,"threads_started":1}
I20260810 06:48:33.090847 10091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.070907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.10.242.3:42803)
0810 06:48:33.083243 (+ 12336us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.083251 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.083288 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:48:33.083392 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.083401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.083637 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.083833 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:33.084132 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.085621 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.087790 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.089591 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.089710 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.089912 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.090133 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.090310 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12165,"thread_start_us":11991,"threads_started":1}
I20260810 06:48:33.091733 10088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.070923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:41775 (local address 127.0.0.1:37528)
0810 06:48:33.081453 (+ 10530us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:33.081471 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:33.081587 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:33.084642 (+ 3055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:33.084646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:33.084671 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:33.085100 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.085115 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.087253 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.087258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.089242 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.089257 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.090999 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.091006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.091179 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.091211 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:48:33.091301 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 06:48:33.091413 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10405,"thread_start_us":10740,"threads_started":1}
I20260810 06:48:33.091827 10095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.071632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.10.242.1:41775)
0810 06:48:33.083455 (+ 11823us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.083463 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.083488 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:48:33.083576 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.083583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.084232 (+ 649us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.084407 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:33.085292 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.087105 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.089419 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.090802 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.090940 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.091168 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.091372 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.091523 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11655,"thread_start_us":12178,"threads_started":1}
I20260810 06:48:33.094655 9952 tablet_service.cc:1467] Processing CreateTablet for tablet f7b2e47538504286bac0082325cf3ec4 (DEFAULT_TABLE table=test-table [id=6c1e500db5df4fdd84b3544d0ee84753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:33.095273 9467 tablet_service.cc:1467] Processing CreateTablet for tablet f7b2e47538504286bac0082325cf3ec4 (DEFAULT_TABLE table=test-table [id=6c1e500db5df4fdd84b3544d0ee84753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:33.096112 9952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b2e47538504286bac0082325cf3ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:33.096614 9467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b2e47538504286bac0082325cf3ec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:33.100917 10097 tablet_bootstrap.cc:492] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: Bootstrap starting.
I20260810 06:48:33.104061 10097 tablet_bootstrap.cc:654] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:33.105229 10097 log.cc:824] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:33.108279 10097 tablet_bootstrap.cc:492] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: No bootstrap required, opened a new log
I20260810 06:48:33.108526 10097 ts_tablet_manager.cc:1397] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260810 06:48:33.113359 10103 tablet_bootstrap.cc:492] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: Bootstrap starting.
I20260810 06:48:33.113600 10102 tablet_bootstrap.cc:492] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: Bootstrap starting.
I20260810 06:48:33.117130 10102 tablet_bootstrap.cc:654] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:33.117709 10097 raft_consensus.cc:348] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } }
I20260810 06:48:33.118145 10097 raft_consensus.cc:374] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:33.118273 10097 raft_consensus.cc:729] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b70c5dc1f63c490d915c7632a8746c4d, State: Initialized, Role: FOLLOWER
I20260810 06:48:33.118906 10097 consensus_queue.cc:260] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } }
I20260810 06:48:33.119956 9822 heartbeater.cc:503] Master 127.10.242.62:32867 was elected leader, sending a full tablet report...
I20260810 06:48:33.120645 10097 ts_tablet_manager.cc:1428] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: Time spent starting tablet: real 0.012s user 0.010s sys 0.003s
I20260810 06:48:33.121199 10102 log.cc:824] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:33.121439 10103 tablet_bootstrap.cc:654] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:33.122509 10103 log.cc:824] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:33.124099 10102 tablet_bootstrap.cc:492] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: No bootstrap required, opened a new log
I20260810 06:48:33.124330 10102 ts_tablet_manager.cc:1397] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.006s
I20260810 06:48:33.126349 10103 tablet_bootstrap.cc:492] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: No bootstrap required, opened a new log
I20260810 06:48:33.126602 10103 ts_tablet_manager.cc:1397] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260810 06:48:33.134668 10102 raft_consensus.cc:348] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } }
I20260810 06:48:33.135167 10102 raft_consensus.cc:374] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:33.135272 10102 raft_consensus.cc:729] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac937c431691477583feac80bb714cff, State: Initialized, Role: FOLLOWER
I20260810 06:48:33.135967 10102 consensus_queue.cc:260] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } }
I20260810 06:48:33.136690 10103 raft_consensus.cc:348] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } }
I20260810 06:48:33.137133 10103 raft_consensus.cc:374] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:33.137235 10103 raft_consensus.cc:729] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a041c8d90345d581ab73675ed36215, State: Initialized, Role: FOLLOWER
I20260810 06:48:33.137910 10102 ts_tablet_manager.cc:1428] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: Time spent starting tablet: real 0.013s user 0.012s sys 0.002s
I20260810 06:48:33.137866 10103 consensus_queue.cc:260] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } }
I20260810 06:48:33.139016 10062 heartbeater.cc:503] Master 127.10.242.62:32867 was elected leader, sending a full tablet report...
I20260810 06:48:33.139549 10103 ts_tablet_manager.cc:1428] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260810 06:48:33.226109 9593 tablet.cc:2369] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:48:33.246140 9824 tablet.cc:2369] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:48:33.255393 10064 tablet.cc:2369] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:33.325708 10105 raft_consensus.cc:482] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:33.325989 10105 raft_consensus.cc:504] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } }
I20260810 06:48:33.327744 10105 leader_election.cc:288] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac937c431691477583feac80bb714cff (127.10.242.1:41775), b3a041c8d90345d581ab73675ed36215 (127.10.242.3:42803)
I20260810 06:48:33.334514 10095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.328151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:46835 (local address 127.10.242.1:41775)
0810 06:48:33.328373 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.328379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.329028 (+ 649us) server_negotiation.cc:407] Connection header received
0810 06:48:33.329231 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.329237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.329330 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.329425 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:33.329991 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.330787 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.332607 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.333379 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.333525 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.333607 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.334114 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.334279 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:48:33.334698 10114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.327931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:41775 (local address 127.10.242.2:46835)
0810 06:48:33.328921 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:33.328936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:33.329071 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:33.329477 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:33.329482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:33.329513 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:33.329826 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.329834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.330915 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.330919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.332446 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.332458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.333584 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.333590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.333934 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.333971 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:48:33.334071 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:48:33.334223 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"mutex_wait_us":38,"thread_start_us":677,"threads_started":1}
I20260810 06:48:33.335093 10115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.328176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:42803 (local address 127.10.242.2:43699)
0810 06:48:33.329616 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:33.329631 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:33.329759 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:33.330060 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:33.330064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:33.330086 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:33.330431 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.330440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.331419 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.331425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.332806 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:33.332819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.334303 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.334311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:33.334476 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.334510 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:48:33.334599 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:48:33.334723 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1310,"mutex_wait_us":4,"thread_start_us":1097,"threads_started":1}
I20260810 06:48:33.335301 10091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.328282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:43699 (local address 127.10.242.3:42803)
0810 06:48:33.328490 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.328496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.329716 (+ 1220us) server_negotiation.cc:407] Connection header received
0810 06:48:33.329886 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.329890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.329957 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.330029 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:33.330567 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.331289 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.332952 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.333694 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.334199 (+ 505us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.334272 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.334909 (+ 637us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.335043 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 06:48:33.335884 9506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b2e47538504286bac0082325cf3ec4" candidate_uuid: "b70c5dc1f63c490d915c7632a8746c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac937c431691477583feac80bb714cff" is_pre_election: true
I20260810 06:48:33.336090 9982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b2e47538504286bac0082325cf3ec4" candidate_uuid: "b70c5dc1f63c490d915c7632a8746c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a041c8d90345d581ab73675ed36215" is_pre_election: true
I20260810 06:48:33.336314 9506 raft_consensus.cc:2454] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70c5dc1f63c490d915c7632a8746c4d in term 0.
I20260810 06:48:33.336477 9982 raft_consensus.cc:2454] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b70c5dc1f63c490d915c7632a8746c4d in term 0.
I20260810 06:48:33.336977 9633 leader_election.cc:302] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac937c431691477583feac80bb714cff, b70c5dc1f63c490d915c7632a8746c4d; no voters:
I20260810 06:48:33.337365 10105 raft_consensus.cc:2781] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:48:33.337489 10105 raft_consensus.cc:482] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:48:33.337589 10105 raft_consensus.cc:3037] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:33.339782 10105 raft_consensus.cc:504] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } }
I20260810 06:48:33.340322 10105 leader_election.cc:288] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [CANDIDATE]: Term 1 election: Requested vote from peers ac937c431691477583feac80bb714cff (127.10.242.1:41775), b3a041c8d90345d581ab73675ed36215 (127.10.242.3:42803)
I20260810 06:48:33.340775 9506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b2e47538504286bac0082325cf3ec4" candidate_uuid: "b70c5dc1f63c490d915c7632a8746c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac937c431691477583feac80bb714cff"
I20260810 06:48:33.340790 9982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b2e47538504286bac0082325cf3ec4" candidate_uuid: "b70c5dc1f63c490d915c7632a8746c4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a041c8d90345d581ab73675ed36215"
I20260810 06:48:33.340984 9506 raft_consensus.cc:3037] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:33.340984 9982 raft_consensus.cc:3037] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:33.344125 9506 raft_consensus.cc:2454] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70c5dc1f63c490d915c7632a8746c4d in term 1.
I20260810 06:48:33.344125 9982 raft_consensus.cc:2454] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b70c5dc1f63c490d915c7632a8746c4d in term 1.
I20260810 06:48:33.344515 9634 leader_election.cc:302] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3a041c8d90345d581ab73675ed36215, b70c5dc1f63c490d915c7632a8746c4d; no voters:
I20260810 06:48:33.344779 10105 raft_consensus.cc:2781] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:33.345170 10105 raft_consensus.cc:686] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 1 LEADER]: Becoming Leader. State: Replica: b70c5dc1f63c490d915c7632a8746c4d, State: Running, Role: LEADER
I20260810 06:48:33.345662 10105 consensus_queue.cc:237] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } }
I20260810 06:48:33.350572 9260 catalog_manager.cc:5495] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d reported cstate change: term changed from 0 to 1, leader changed from <none> to b70c5dc1f63c490d915c7632a8746c4d (127.10.242.2). New cstate: current_term: 1 leader_uuid: "b70c5dc1f63c490d915c7632a8746c4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b70c5dc1f63c490d915c7632a8746c4d" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 43961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 } health_report { overall_health: UNKNOWN } } }
I20260810 06:48:33.385546 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.380192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55438 (local address 127.10.242.2:43961)
0810 06:48:33.380382 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.380387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.380406 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:48:33.380548 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.380551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.380632 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.380757 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:48:33.381159 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.381886 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.383340 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.383733 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.383837 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.384393 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:48:33.384954 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:48:33.385158 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.385270 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.385311 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:48:33.393546 9982 raft_consensus.cc:1261] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [term 1 FOLLOWER]: Refusing update from remote peer b70c5dc1f63c490d915c7632a8746c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:33.393545 9506 raft_consensus.cc:1261] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [term 1 FOLLOWER]: Refusing update from remote peer b70c5dc1f63c490d915c7632a8746c4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:33.394596 10116 consensus_queue.cc:1035] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3a041c8d90345d581ab73675ed36215" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:33.394927 10105 consensus_queue.cc:1035] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac937c431691477583feac80bb714cff" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 41775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:33.405045 10125 mvcc.cc:205] Tried to move back new op lower bound from 7316867126849683456 to 7316867126667739136. Current Snapshot: MvccSnapshot[applied={T|T < 7316867126849683456}]
I20260810 06:48:33.409459 10127 mvcc.cc:205] Tried to move back new op lower bound from 7316867126849683456 to 7316867126667739136. Current Snapshot: MvccSnapshot[applied={T|T < 7316867126849683456}]
I20260810 06:48:33.413015 10126 mvcc.cc:205] Tried to move back new op lower bound from 7316867126849683456 to 7316867126667739136. Current Snapshot: MvccSnapshot[applied={T|T < 7316867126849683456 or (T in {7316867126849683456})}]
I20260810 06:48:33.418326 9260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:33.418613 9260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:33.422055 9260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 005ae545f7be407da9e82a101d9d8b24: Sending DeleteTablet for 3 replicas of tablet f7b2e47538504286bac0082325cf3ec4
I20260810 06:48:33.423210 9724 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b2e47538504286bac0082325cf3ec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:33 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:55434
I20260810 06:48:33.423235 9952 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b2e47538504286bac0082325cf3ec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:33 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:47468
I20260810 06:48:33.423911 9467 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b2e47538504286bac0082325cf3ec4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:33 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:37528
I20260810 06:48:33.424132 10144 tablet_replica.cc:330] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: stopping tablet replica
I20260810 06:48:33.424592 10144 raft_consensus.cc:2229] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:48:33.425307 10144 raft_consensus.cc:2258] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:33.425706 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 9358
I20260810 06:48:33.426205 10149 tablet_replica.cc:330] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: stopping tablet replica
I20260810 06:48:33.427990 10144 ts_tablet_manager.cc:1905] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:33.435745 10144 ts_tablet_manager.cc:1918] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:33.436275 10149 raft_consensus.cc:2229] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:33.436867 10149 raft_consensus.cc:2258] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:33.437332 10144 log.cc:1196] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/f7b2e47538504286bac0082325cf3ec4
I20260810 06:48:33.439335 10149 ts_tablet_manager.cc:1905] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:33.442814 10144 ts_tablet_manager.cc:1939] T f7b2e47538504286bac0082325cf3ec4 P b70c5dc1f63c490d915c7632a8746c4d: Deleting consensus metadata
I20260810 06:48:33.444559 10147 tablet_replica.cc:330] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: stopping tablet replica
I20260810 06:48:33.444955 10147 raft_consensus.cc:2229] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:33.445421 10147 raft_consensus.cc:2258] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:33.448647 9235 catalog_manager.cc:4841] TS b70c5dc1f63c490d915c7632a8746c4d (127.10.242.2:43961): tablet f7b2e47538504286bac0082325cf3ec4 (table test-table [id=6c1e500db5df4fdd84b3544d0ee84753]) successfully deleted
I20260810 06:48:33.448807 10147 ts_tablet_manager.cc:1905] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:33.449023 10149 ts_tablet_manager.cc:1918] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:33.449188 10149 log.cc:1196] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/f7b2e47538504286bac0082325cf3ec4
I20260810 06:48:33.449693 10149 ts_tablet_manager.cc:1939] T f7b2e47538504286bac0082325cf3ec4 P ac937c431691477583feac80bb714cff: Deleting consensus metadata
I20260810 06:48:33.451355 9235 catalog_manager.cc:4841] TS ac937c431691477583feac80bb714cff (127.10.242.1:41775): tablet f7b2e47538504286bac0082325cf3ec4 (table test-table [id=6c1e500db5df4fdd84b3544d0ee84753]) successfully deleted
I20260810 06:48:33.453972 10147 ts_tablet_manager.cc:1918] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:33.454109 10147 log.cc:1196] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/f7b2e47538504286bac0082325cf3ec4
I20260810 06:48:33.454630 10147 ts_tablet_manager.cc:1939] T f7b2e47538504286bac0082325cf3ec4 P b3a041c8d90345d581ab73675ed36215: Deleting consensus metadata
I20260810 06:48:33.456256 9237 catalog_manager.cc:4841] TS b3a041c8d90345d581ab73675ed36215 (127.10.242.3:42803): tablet f7b2e47538504286bac0082325cf3ec4 (table test-table [id=6c1e500db5df4fdd84b3544d0ee84753]) successfully deleted
Aug 10 06:48:33 dist-test-slave-zj8q krb5kdc[9187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:48:33.464111 10095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.427321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37530 (local address 127.10.242.1:41775)
0810 06:48:33.427489 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.427495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.438056 (+ 10561us) server_negotiation.cc:407] Connection header received
0810 06:48:33.438528 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.438533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.438625 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.438697 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:33.440549 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.441713 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.445831 (+ 4118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.446524 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.446637 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.458613 (+ 11976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:33.458624 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:33.458638 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:33.458670 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:33.461419 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.462281 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.462288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.462290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.462420 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.462795 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.462798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.462800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.463156 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:33.463378 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.463695 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.463852 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:48:33.464838 9582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:33.552145 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 9358
I20260810 06:48:33.579969 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 9603
Aug 10 06:48:33 dist-test-slave-zj8q krb5kdc[9187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:48:33.597579 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.580858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.10.242.2:43961)
0810 06:48:33.581082 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.581089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.581117 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:48:33.581185 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.581190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.581277 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.581403 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:33.582825 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.584071 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.585395 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.586128 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.586284 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.590729 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:33.590749 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:33.590826 (+ 77us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:33.590876 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:33.594812 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.595446 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.595457 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.595460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.595625 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.595935 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.595940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.595943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.596421 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:33.596768 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.596929 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.597187 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:48:33.598398 9813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:33.708479 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 9603
I20260810 06:48:33.731334 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 9836
Aug 10 06:48:33 dist-test-slave-zj8q krb5kdc[9187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:48:33.745368 10091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:33.732057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47482 (local address 127.10.242.3:42803)
0810 06:48:33.732248 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:33.732253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:33.732273 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:48:33.732397 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:33.732401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:33.732465 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:33.732544 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:33.733834 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.734622 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.735848 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:33.736212 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:33.736297 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:33.740532 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:33.740545 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:33.740562 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:33.740602 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:33.743064 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.743701 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.743708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.743710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.743855 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:33.744153 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:33.744156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:33.744158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:33.744511 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:33.744721 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:33.744949 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 06:48:33.745109 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:48:33.745985 10055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:33.832222 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 9836
I20260810 06:48:33.854684 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 9212
I20260810 06:48:33.855774 9307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:33.963101 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 9212
2026-08-10T06:48:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4583 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T06:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:34Z Disabled control of system clock
I20260810 06:48:34.055256 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:36687
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:33765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:36687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:48:34.383737 10166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:34.384003 10166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:34.384083 10166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:34.391222 10166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:34.391321 10166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:34.391366 10166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:34.391420 10166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:34.391469 10166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:34.400197 10166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:36687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:36687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:34.401335 10166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:34.403095 10166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:34.414206 10178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:34.414274 10184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:34.414225 10180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:34.415295 10166 server_base.cc:1034] running on GCE node
I20260810 06:48:34.416014 10166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:34.417315 10166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:34.418503 10166 hybrid_clock.cc:648] HybridClock initialized: now 1786344514418441 us; error 56 us; skew 500 ppm
I20260810 06:48:34.418833 10166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:34.421016 10166 webserver.cc:460] Webserver started at http://127.10.242.62:32837/ using document root <none> and password file <none>
I20260810 06:48:34.421644 10166 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:34.421759 10166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:34.422060 10166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:34.424846 10166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "a23cc521c5ed40a5aa84d1d000b1eccc"
format_stamp: "Formatted at 2026-08-10 06:48:34 on dist-test-slave-zj8q"
server_key: "43317530013d43b61bc078ea79f2e6f7"
server_key_iv: "3579e0bd89195ce6dcfc0cde170b3c59"
server_key_version: "encryptionkey@0"
I20260810 06:48:34.425616 10166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "a23cc521c5ed40a5aa84d1d000b1eccc"
format_stamp: "Formatted at 2026-08-10 06:48:34 on dist-test-slave-zj8q"
server_key: "43317530013d43b61bc078ea79f2e6f7"
server_key_iv: "3579e0bd89195ce6dcfc0cde170b3c59"
server_key_version: "encryptionkey@0"
I20260810 06:48:34.430454 10166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 06:48:34.433643 10192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:34.434861 10166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:48:34.435016 10166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "a23cc521c5ed40a5aa84d1d000b1eccc"
format_stamp: "Formatted at 2026-08-10 06:48:34 on dist-test-slave-zj8q"
server_key: "43317530013d43b61bc078ea79f2e6f7"
server_key_iv: "3579e0bd89195ce6dcfc0cde170b3c59"
server_key_version: "encryptionkey@0"
I20260810 06:48:34.435197 10166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:34.456712 10166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:34.457420 10166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:34.457664 10166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:34.473750 10166 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:36687
I20260810 06:48:34.473781 10291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:36687 every 8 connection(s)
I20260810 06:48:34.474982 10166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:34.477658 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 10166
I20260810 06:48:34.477816 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:34.478292 11208 external_mini_cluster.cc:1428] Setting key 691b5f1a2b17699c31ea52c053d8ccdd
I20260810 06:48:34.478595 10292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:34.493057 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:34.480076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39786 (local address 127.10.242.62:36687)
0810 06:48:34.481147 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:34.481165 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:34.481215 (+ 50us) server_negotiation.cc:407] Connection header received
0810 06:48:34.483355 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:34.483386 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:34.483973 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:34.484398 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:34.484946 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:34.486993 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:34.488276 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:34.489262 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:34.489439 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:34.490459 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:34.490470 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:34.490487 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:34.491181 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:34.491280 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:34.491323 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:34.491439 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:34.491562 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 06:48:34.491857 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":337,"threads_started":1}
I20260810 06:48:34.493497 10292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc: Bootstrap starting.
I20260810 06:48:34.496905 10292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:34.497851 10292 log.cc:824] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:34.500417 10292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc: No bootstrap required, opened a new log
I20260810 06:48:34.507822 10292 raft_consensus.cc:348] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23cc521c5ed40a5aa84d1d000b1eccc" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36687 } }
I20260810 06:48:34.508140 10292 raft_consensus.cc:374] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:34.508212 10292 raft_consensus.cc:729] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23cc521c5ed40a5aa84d1d000b1eccc, State: Initialized, Role: FOLLOWER
I20260810 06:48:34.508654 10292 consensus_queue.cc:260] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23cc521c5ed40a5aa84d1d000b1eccc" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36687 } }
I20260810 06:48:34.508859 10292 raft_consensus.cc:388] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:34.508955 10292 raft_consensus.cc:482] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:34.509066 10292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:34.511058 10292 raft_consensus.cc:504] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23cc521c5ed40a5aa84d1d000b1eccc" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36687 } }
I20260810 06:48:34.511499 10292 leader_election.cc:302] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a23cc521c5ed40a5aa84d1d000b1eccc; no voters:
I20260810 06:48:34.511986 10292 leader_election.cc:288] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:34.512110 10302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:34.513134 10302 raft_consensus.cc:686] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [term 1 LEADER]: Becoming Leader. State: Replica: a23cc521c5ed40a5aa84d1d000b1eccc, State: Running, Role: LEADER
I20260810 06:48:34.513696 10302 consensus_queue.cc:237] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23cc521c5ed40a5aa84d1d000b1eccc" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36687 } }
I20260810 06:48:34.513883 10292 sys_catalog.cc:564] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:34.516306 10304 sys_catalog.cc:455] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a23cc521c5ed40a5aa84d1d000b1eccc. Latest consensus state: current_term: 1 leader_uuid: "a23cc521c5ed40a5aa84d1d000b1eccc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23cc521c5ed40a5aa84d1d000b1eccc" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36687 } } }
I20260810 06:48:34.516700 10304 sys_catalog.cc:458] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:34.517172 10303 sys_catalog.cc:455] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a23cc521c5ed40a5aa84d1d000b1eccc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23cc521c5ed40a5aa84d1d000b1eccc" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 36687 } } }
I20260810 06:48:34.517478 10303 sys_catalog.cc:458] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:34.518378 10319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:34.523430 10319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:34.531780 10319 catalog_manager.cc:1347] Generated new cluster ID: 95e442090e4942cba0875a87fa671350
I20260810 06:48:34.531876 10319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:34.544538 10319 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:34.545717 10319 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:34.555997 10319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc: Generated new TSK 0
I20260810 06:48:34.556703 10319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:34.564554 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36687
--builtin_ntp_servers=127.10.242.20:33765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:48:34.897795 10333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:34.898051 10333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:34.898200 10333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:34.905166 10333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:34.905258 10333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:34.905403 10333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:48:34.914484 10333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:34.915640 10333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:34.917339 10333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:34.929908 10347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:34.929945 10343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:34.930480 10342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:34.930781 10333 server_base.cc:1034] running on GCE node
I20260810 06:48:34.931331 10333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:34.932144 10333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:34.933347 10333 hybrid_clock.cc:648] HybridClock initialized: now 1786344514933306 us; error 28 us; skew 500 ppm
I20260810 06:48:34.933610 10333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:34.938877 10333 webserver.cc:460] Webserver started at http://127.10.242.1:43119/ using document root <none> and password file <none>
I20260810 06:48:34.939546 10333 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:34.939651 10333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:34.939917 10333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:34.942737 10333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "a2f35195cc634c6b835a1f0abeca479f"
format_stamp: "Formatted at 2026-08-10 06:48:34 on dist-test-slave-zj8q"
server_key: "6d503e7804bb658b0db43fce6d6972ea"
server_key_iv: "5f767b57451aecd10d08c6be3f6f491b"
server_key_version: "encryptionkey@0"
I20260810 06:48:34.943485 10333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "a2f35195cc634c6b835a1f0abeca479f"
format_stamp: "Formatted at 2026-08-10 06:48:34 on dist-test-slave-zj8q"
server_key: "6d503e7804bb658b0db43fce6d6972ea"
server_key_iv: "5f767b57451aecd10d08c6be3f6f491b"
server_key_version: "encryptionkey@0"
I20260810 06:48:34.948408 10333 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:48:34.951772 10359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:34.952971 10333 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:48:34.953099 10333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "a2f35195cc634c6b835a1f0abeca479f"
format_stamp: "Formatted at 2026-08-10 06:48:34 on dist-test-slave-zj8q"
server_key: "6d503e7804bb658b0db43fce6d6972ea"
server_key_iv: "5f767b57451aecd10d08c6be3f6f491b"
server_key_version: "encryptionkey@0"
I20260810 06:48:34.953269 10333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:34.974113 10333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:34.974833 10333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:34.975083 10333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:34.975922 10333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:34.977241 10333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:34.977324 10333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:34.977397 10333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:34.977450 10333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:35.006985 10333 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:43625
I20260810 06:48:35.007071 10539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:43625 every 8 connection(s)
I20260810 06:48:35.008220 10333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:35.015599 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 10333
I20260810 06:48:35.015750 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:48:35.016184 11208 external_mini_cluster.cc:1428] Setting key 477a14522e914fa1279e15e4474358c0
I20260810 06:48:35.019050 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36687
--builtin_ntp_servers=127.10.242.20:33765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:48:35.020533 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.010705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:44341 (local address 127.10.242.62:36687)
0810 06:48:35.011122 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:35.011128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:35.012265 (+ 1137us) server_negotiation.cc:407] Connection header received
0810 06:48:35.014013 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:35.014018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:35.014099 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:35.014231 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:35.015257 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.016032 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.017731 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.018377 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.018509 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.019775 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:35.019781 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:35.019785 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:35.019804 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:35.019859 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:35.019875 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:35.019968 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:35.020127 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 06:48:35.020219 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":206,"threads_started":1}
I20260810 06:48:35.021349 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.011269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:36687 (local address 127.10.242.1:44341)
0810 06:48:35.012114 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:35.012157 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:35.013754 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:35.014453 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:35.014460 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:35.014550 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:35.015087 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.015103 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.016196 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.016200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.017571 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.017584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.018582 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.018588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.018804 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.019454 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:48:35.019457 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:35.019494 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:48:35.019571 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:35.019961 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:35.019967 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:35.019972 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:48:35.020098 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 06:48:35.020369 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":258,"threads_started":1}
I20260810 06:48:35.022897 10541 heartbeater.cc:347] Connected to a master server at 127.10.242.62:36687
I20260810 06:48:35.023175 10541 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:35.023902 10541 heartbeater.cc:511] Master 127.10.242.62:36687 requested a full tablet report, sending...
I20260810 06:48:35.025799 10223 ts_manager.cc:194] Registered new tserver with Master: a2f35195cc634c6b835a1f0abeca479f (127.10.242.1:43625)
I20260810 06:48:35.027268 10223 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.1:44341
W20260810 06:48:35.349157 10545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:35.349414 10545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:35.349545 10545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:35.356488 10545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:35.356577 10545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:35.356725 10545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:48:35.365428 10545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:35.366636 10545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:35.368358 10545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:35.380532 10556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:35.380501 10561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:35.380473 10557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:35.381265 10545 server_base.cc:1034] running on GCE node
I20260810 06:48:35.381892 10545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:35.382810 10545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:35.384011 10545 hybrid_clock.cc:648] HybridClock initialized: now 1786344515383959 us; error 44 us; skew 500 ppm
I20260810 06:48:35.384311 10545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:35.386380 10545 webserver.cc:460] Webserver started at http://127.10.242.2:41551/ using document root <none> and password file <none>
I20260810 06:48:35.387009 10545 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:35.387110 10545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:35.387434 10545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:35.390225 10545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "e13e424a6bb34e7cb816ebbfba174606"
format_stamp: "Formatted at 2026-08-10 06:48:35 on dist-test-slave-zj8q"
server_key: "bf22ad752a2a4b66a809d7f1a026a2b0"
server_key_iv: "425db475949d227835fbf410fc24d68e"
server_key_version: "encryptionkey@0"
I20260810 06:48:35.390972 10545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "e13e424a6bb34e7cb816ebbfba174606"
format_stamp: "Formatted at 2026-08-10 06:48:35 on dist-test-slave-zj8q"
server_key: "bf22ad752a2a4b66a809d7f1a026a2b0"
server_key_iv: "425db475949d227835fbf410fc24d68e"
server_key_version: "encryptionkey@0"
I20260810 06:48:35.395607 10545 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 06:48:35.398787 10576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:35.399924 10545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:48:35.400075 10545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "e13e424a6bb34e7cb816ebbfba174606"
format_stamp: "Formatted at 2026-08-10 06:48:35 on dist-test-slave-zj8q"
server_key: "bf22ad752a2a4b66a809d7f1a026a2b0"
server_key_iv: "425db475949d227835fbf410fc24d68e"
server_key_version: "encryptionkey@0"
I20260810 06:48:35.400271 10545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:35.412992 10545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:35.413686 10545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:35.413967 10545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:35.414884 10545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:35.416240 10545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:35.416335 10545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:35.416442 10545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:35.416512 10545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:35.445338 10545 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:36011
I20260810 06:48:35.445389 10772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:36011 every 8 connection(s)
I20260810 06:48:35.446623 10545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:48:35.447383 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 10545
I20260810 06:48:35.447559 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:48:35.448076 11208 external_mini_cluster.cc:1428] Setting key 9508875f0000614c8223fddb8a0c889a
I20260810 06:48:35.451290 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36687
--builtin_ntp_servers=127.10.242.20:33765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:48:35.460026 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.449256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:58261 (local address 127.10.242.62:36687)
0810 06:48:35.449453 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:35.449461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:35.451163 (+ 1702us) server_negotiation.cc:407] Connection header received
0810 06:48:35.453413 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:35.453417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:35.453486 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:35.453567 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:35.454795 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.455477 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.457513 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.457923 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.458015 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.459386 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:35.459391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:35.459394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:35.459411 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:35.459441 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:35.459453 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:35.459524 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:35.459740 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 06:48:35.459812 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:48:35.461190 10778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.449875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:36687 (local address 127.10.242.2:58261)
0810 06:48:35.450971 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:35.451049 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:35.453145 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:35.453808 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:35.453818 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:35.453917 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:35.454639 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.454662 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.455628 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.455633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.457337 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.457352 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.458123 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.458129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.458411 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.459075 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:48:35.459079 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:35.459128 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:48:35.459215 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:35.459543 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:35.459552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:35.459560 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:48:35.459724 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 06:48:35.460064 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":341,"threads_started":1}
I20260810 06:48:35.462793 10775 heartbeater.cc:347] Connected to a master server at 127.10.242.62:36687
I20260810 06:48:35.463052 10775 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:35.463732 10775 heartbeater.cc:511] Master 127.10.242.62:36687 requested a full tablet report, sending...
I20260810 06:48:35.465060 10223 ts_manager.cc:194] Registered new tserver with Master: e13e424a6bb34e7cb816ebbfba174606 (127.10.242.2:36011)
I20260810 06:48:35.465852 10223 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.2:58261
W20260810 06:48:35.784554 10779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:35.784806 10779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:35.784925 10779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:35.791888 10779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:35.791976 10779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:35.792112 10779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:48:35.800655 10779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:36687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:35.801781 10779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:35.803603 10779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:35.816012 10790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:35.816005 10792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:35.816159 10795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:35.816762 10779 server_base.cc:1034] running on GCE node
I20260810 06:48:35.817337 10779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:35.818269 10779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:35.819464 10779 hybrid_clock.cc:648] HybridClock initialized: now 1786344515819416 us; error 40 us; skew 500 ppm
I20260810 06:48:35.819756 10779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:35.821821 10779 webserver.cc:460] Webserver started at http://127.10.242.3:40367/ using document root <none> and password file <none>
I20260810 06:48:35.822487 10779 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:35.822588 10779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:35.822902 10779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:35.825651 10779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "db7fcc0e77d74821bcc8a398df31edc2"
format_stamp: "Formatted at 2026-08-10 06:48:35 on dist-test-slave-zj8q"
server_key: "de8047cbae7b8ea47b0c6e327e4b7b5b"
server_key_iv: "47bcc40ed03e4768f53d50d78326c8b7"
server_key_version: "encryptionkey@0"
I20260810 06:48:35.826504 10779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "db7fcc0e77d74821bcc8a398df31edc2"
format_stamp: "Formatted at 2026-08-10 06:48:35 on dist-test-slave-zj8q"
server_key: "de8047cbae7b8ea47b0c6e327e4b7b5b"
server_key_iv: "47bcc40ed03e4768f53d50d78326c8b7"
server_key_version: "encryptionkey@0"
I20260810 06:48:35.831521 10779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:48:35.834950 10806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:35.836225 10779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:48:35.836370 10779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "db7fcc0e77d74821bcc8a398df31edc2"
format_stamp: "Formatted at 2026-08-10 06:48:35 on dist-test-slave-zj8q"
server_key: "de8047cbae7b8ea47b0c6e327e4b7b5b"
server_key_iv: "47bcc40ed03e4768f53d50d78326c8b7"
server_key_version: "encryptionkey@0"
I20260810 06:48:35.836565 10779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:35.850760 10779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:35.851464 10779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:35.851725 10779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:35.852634 10779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:35.853952 10779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:35.854045 10779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:35.854162 10779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:35.854240 10779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260810 06:48:35.882004 10779 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:34093
I20260810 06:48:35.882050 10969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:34093 every 8 connection(s)
I20260810 06:48:35.883363 10779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:48:35.892782 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 10779
I20260810 06:48:35.892959 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:48:35.893350 11208 external_mini_cluster.cc:1428] Setting key f4aa6de18451a48e5126441854615171
I20260810 06:48:35.895206 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.885816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:56035 (local address 127.10.242.62:36687)
0810 06:48:35.886015 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:35.886020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:35.887354 (+ 1334us) server_negotiation.cc:407] Connection header received
0810 06:48:35.889065 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:35.889070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:35.889149 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:35.889234 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:35.890336 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.891059 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.892572 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.893011 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.893143 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.894397 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:35.894404 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:35.894408 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:35.894428 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:35.894466 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:35.894484 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:35.894571 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:35.894797 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 06:48:35.894899 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:48:35.896243 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.886339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:36687 (local address 127.10.242.3:56035)
0810 06:48:35.887201 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:35.887243 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:35.888801 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:35.889469 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:35.889477 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:35.889565 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:35.890144 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.890182 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.891208 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.891212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.892413 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.892426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.893238 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.893247 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.893462 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.894067 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:48:35.894069 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:35.894106 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:48:35.894218 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:35.894585 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:35.894596 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:35.894605 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:48:35.894738 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 06:48:35.895068 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":265,"threads_started":1}
I20260810 06:48:35.897375 10972 heartbeater.cc:347] Connected to a master server at 127.10.242.62:36687
I20260810 06:48:35.897609 10972 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:35.898321 10972 heartbeater.cc:511] Master 127.10.242.62:36687 requested a full tablet report, sending...
I20260810 06:48:35.899504 10223 ts_manager.cc:194] Registered new tserver with Master: db7fcc0e77d74821bcc8a398df31edc2 (127.10.242.3:34093)
I20260810 06:48:35.900245 10223 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.3:56035
I20260810 06:48:35.908056 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:48:35.917296 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.911923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.10.242.62:36687)
0810 06:48:35.912060 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:35.912064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:35.912524 (+ 460us) server_negotiation.cc:407] Connection header received
0810 06:48:35.912643 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:35.912646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:35.912714 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:35.912788 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:35.913224 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.913897 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.915363 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.915786 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.915884 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.916768 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:35.916773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:35.916776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:35.916791 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:35.916821 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:35.916831 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:35.916932 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:35.917037 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 06:48:35.917090 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:48:35.920785 10223 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:48:35.928785 10223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:48:35.949333 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.939919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:43625 (local address 127.0.0.1:56540)
0810 06:48:35.940933 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:35.940975 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:35.941157 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:35.943881 (+ 2724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:35.943888 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:35.943919 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:35.944319 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.944331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.945608 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.945614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.947161 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.947178 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.948486 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.948492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.948690 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.948738 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:48:35.948850 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:48:35.949027 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"threads_started":1}
I20260810 06:48:35.949380 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.940154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56540 (local address 127.10.242.1:43625)
0810 06:48:35.943386 (+ 3232us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:35.943392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:35.943416 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:48:35.943496 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:35.943501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:35.943685 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:35.943849 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:35.944487 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.945471 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.947345 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.948307 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.948424 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.948676 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:35.948901 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:48:35.949064 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3112,"thread_start_us":2999,"threads_started":1}
I20260810 06:48:35.954664 10430 tablet_service.cc:1467] Processing CreateTablet for tablet 04c2e5e4140c4e5caf543b262148d646 (DEFAULT_TABLE table=test-table [id=68731ab4b9b54254911c86f2c278f448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:35.956151 10430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04c2e5e4140c4e5caf543b262148d646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:35.956691 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.940176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:34093 (local address 127.0.0.1:41626)
0810 06:48:35.949569 (+ 9393us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:35.949589 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:35.949701 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:35.950553 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:35.950556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:35.950576 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:35.950701 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.950709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.952498 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.952503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.954323 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.954341 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.955973 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.955980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.956145 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.956188 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 06:48:35.956275 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:48:35.956395 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9272,"thread_start_us":494,"threads_started":1}
I20260810 06:48:35.957396 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.940494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.10.242.3:34093)
0810 06:48:35.943151 (+ 2657us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:35.943162 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:35.949836 (+ 6674us) server_negotiation.cc:407] Connection header received
0810 06:48:35.949925 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:35.949930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:35.950088 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:35.950299 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:35.950880 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.952354 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.954481 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.955812 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.955942 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.956198 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:35.956926 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 06:48:35.957079 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2484,"thread_start_us":2355,"threads_started":1}
I20260810 06:48:35.957408 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.950084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:36011 (local address 127.0.0.1:49384)
0810 06:48:35.950311 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:35.950327 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:35.950424 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:35.951581 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:35.951586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:35.951613 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:35.951926 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.951937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.953596 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.953601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.955173 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:35.955187 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.956762 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.956769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:35.956945 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.956975 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:48:35.957035 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 06:48:35.957160 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 06:48:35.957651 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:35.950500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49384 (local address 127.10.242.2:36011)
0810 06:48:35.951055 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:35.951063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:35.951089 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:48:35.951187 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:35.951194 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:35.951392 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:35.951574 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:35.952072 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.953449 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.955380 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:35.956533 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:35.956650 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:35.956996 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:35.957094 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 06:48:35.957242 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":260,"threads_started":1}
I20260810 06:48:35.961005 10866 tablet_service.cc:1467] Processing CreateTablet for tablet 04c2e5e4140c4e5caf543b262148d646 (DEFAULT_TABLE table=test-table [id=68731ab4b9b54254911c86f2c278f448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:35.961005 10659 tablet_service.cc:1467] Processing CreateTablet for tablet 04c2e5e4140c4e5caf543b262148d646 (DEFAULT_TABLE table=test-table [id=68731ab4b9b54254911c86f2c278f448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:35.962436 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04c2e5e4140c4e5caf543b262148d646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:35.962442 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04c2e5e4140c4e5caf543b262148d646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:35.969266 11001 tablet_bootstrap.cc:492] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: Bootstrap starting.
I20260810 06:48:35.972237 11001 tablet_bootstrap.cc:654] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:35.973244 11001 log.cc:824] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:35.976425 11001 tablet_bootstrap.cc:492] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: No bootstrap required, opened a new log
I20260810 06:48:35.976687 11004 tablet_bootstrap.cc:492] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: Bootstrap starting.
I20260810 06:48:35.976770 11001 ts_tablet_manager.cc:1397] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260810 06:48:35.978848 11005 tablet_bootstrap.cc:492] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: Bootstrap starting.
I20260810 06:48:35.980003 11004 tablet_bootstrap.cc:654] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:35.981220 11004 log.cc:824] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:35.981899 11005 tablet_bootstrap.cc:654] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:35.983026 11005 log.cc:824] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:35.984220 11004 tablet_bootstrap.cc:492] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: No bootstrap required, opened a new log
I20260810 06:48:35.984472 11004 ts_tablet_manager.cc:1397] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 06:48:35.985344 11005 tablet_bootstrap.cc:492] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: No bootstrap required, opened a new log
I20260810 06:48:35.985581 11005 ts_tablet_manager.cc:1397] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 06:48:35.986936 11001 raft_consensus.cc:348] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } }
I20260810 06:48:35.987404 11001 raft_consensus.cc:374] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:35.987505 11001 raft_consensus.cc:729] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2f35195cc634c6b835a1f0abeca479f, State: Initialized, Role: FOLLOWER
I20260810 06:48:35.988092 11001 consensus_queue.cc:260] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } }
I20260810 06:48:35.989754 11001 ts_tablet_manager.cc:1428] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: Time spent starting tablet: real 0.013s user 0.010s sys 0.001s
I20260810 06:48:35.990244 10541 heartbeater.cc:503] Master 127.10.242.62:36687 was elected leader, sending a full tablet report...
I20260810 06:48:35.993330 11004 raft_consensus.cc:348] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } }
I20260810 06:48:35.993637 11004 raft_consensus.cc:374] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:35.993741 11004 raft_consensus.cc:729] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e13e424a6bb34e7cb816ebbfba174606, State: Initialized, Role: FOLLOWER
I20260810 06:48:35.993623 11005 raft_consensus.cc:348] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } }
I20260810 06:48:35.994087 11005 raft_consensus.cc:374] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:35.994230 11005 raft_consensus.cc:729] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db7fcc0e77d74821bcc8a398df31edc2, State: Initialized, Role: FOLLOWER
I20260810 06:48:35.994424 11004 consensus_queue.cc:260] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } }
I20260810 06:48:35.994750 11005 consensus_queue.cc:260] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } }
I20260810 06:48:35.995460 10775 heartbeater.cc:503] Master 127.10.242.62:36687 was elected leader, sending a full tablet report...
I20260810 06:48:35.996088 11004 ts_tablet_manager.cc:1428] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260810 06:48:35.997216 10972 heartbeater.cc:503] Master 127.10.242.62:36687 was elected leader, sending a full tablet report...
I20260810 06:48:35.997772 11005 ts_tablet_manager.cc:1428] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260810 06:48:36.010371 10542 tablet.cc:2369] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:36.122020 11011 raft_consensus.cc:482] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:36.122395 11011 raft_consensus.cc:504] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } }
I20260810 06:48:36.124583 11011 leader_election.cc:288] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2f35195cc634c6b835a1f0abeca479f (127.10.242.1:43625), db7fcc0e77d74821bcc8a398df31edc2 (127.10.242.3:34093)
I20260810 06:48:36.131482 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:36.125181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:52981 (local address 127.10.242.1:43625)
0810 06:48:36.125387 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:36.125394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:36.126207 (+ 813us) server_negotiation.cc:407] Connection header received
0810 06:48:36.126355 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:36.126365 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:36.126453 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:36.126535 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:36.127151 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.127867 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.129796 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.130611 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.130759 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:36.130829 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:36.131103 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 06:48:36.131234 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:48:36.131516 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:36.125174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:32791 (local address 127.10.242.3:34093)
0810 06:48:36.125393 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:36.125399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:36.125796 (+ 397us) server_negotiation.cc:407] Connection header received
0810 06:48:36.126004 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:36.126010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:36.126075 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:36.126167 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:36.127059 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.127793 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.129824 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.130679 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.130830 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:36.130898 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:36.131172 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 06:48:36.131290 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:48:36.131577 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:36.124937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:43625 (local address 127.10.242.2:52981)
0810 06:48:36.126073 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:36.126088 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:36.126236 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:36.126558 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:36.126562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:36.126586 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:36.126996 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:36.127006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.128002 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.128007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:36.129603 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:36.129616 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.130805 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.130811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:36.130959 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:36.130986 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 06:48:36.131064 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:48:36.131188 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1042,"mutex_wait_us":39,"thread_start_us":438,"threads_started":1}
I20260810 06:48:36.131623 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:36.124815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:34093 (local address 127.10.242.2:32791)
0810 06:48:36.125693 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:36.125707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:36.125863 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:36.126209 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:36.126213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:36.126239 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:36.126911 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:36.126922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.127916 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.127923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:36.129620 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:36.129633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.130859 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.130865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:36.131025 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:36.131055 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:48:36.131129 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:48:36.131251 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":107,"thread_start_us":972,"threads_started":1}
I20260810 06:48:36.132632 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c2e5e4140c4e5caf543b262148d646" candidate_uuid: "e13e424a6bb34e7cb816ebbfba174606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2f35195cc634c6b835a1f0abeca479f" is_pre_election: true
I20260810 06:48:36.132885 10895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c2e5e4140c4e5caf543b262148d646" candidate_uuid: "e13e424a6bb34e7cb816ebbfba174606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db7fcc0e77d74821bcc8a398df31edc2" is_pre_election: true
I20260810 06:48:36.133140 10463 raft_consensus.cc:2454] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e13e424a6bb34e7cb816ebbfba174606 in term 0.
I20260810 06:48:36.133359 10895 raft_consensus.cc:2454] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e13e424a6bb34e7cb816ebbfba174606 in term 0.
I20260810 06:48:36.133834 10582 leader_election.cc:302] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2f35195cc634c6b835a1f0abeca479f, e13e424a6bb34e7cb816ebbfba174606; no voters:
I20260810 06:48:36.134248 11011 raft_consensus.cc:2781] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:48:36.134375 11011 raft_consensus.cc:482] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:48:36.134472 11011 raft_consensus.cc:3037] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 0 FOLLOWER]: Advancing to term 1
W20260810 06:48:36.134809 10974 tablet.cc:2369] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:36.136624 11011 raft_consensus.cc:504] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } }
I20260810 06:48:36.137199 11011 leader_election.cc:288] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [CANDIDATE]: Term 1 election: Requested vote from peers a2f35195cc634c6b835a1f0abeca479f (127.10.242.1:43625), db7fcc0e77d74821bcc8a398df31edc2 (127.10.242.3:34093)
I20260810 06:48:36.137621 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c2e5e4140c4e5caf543b262148d646" candidate_uuid: "e13e424a6bb34e7cb816ebbfba174606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2f35195cc634c6b835a1f0abeca479f"
I20260810 06:48:36.137634 10895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c2e5e4140c4e5caf543b262148d646" candidate_uuid: "e13e424a6bb34e7cb816ebbfba174606" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db7fcc0e77d74821bcc8a398df31edc2"
I20260810 06:48:36.137800 10463 raft_consensus.cc:3037] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:36.137814 10895 raft_consensus.cc:3037] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:36.140429 10895 raft_consensus.cc:2454] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e13e424a6bb34e7cb816ebbfba174606 in term 1.
I20260810 06:48:36.140586 10463 raft_consensus.cc:2454] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e13e424a6bb34e7cb816ebbfba174606 in term 1.
I20260810 06:48:36.140761 10585 leader_election.cc:302] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db7fcc0e77d74821bcc8a398df31edc2, e13e424a6bb34e7cb816ebbfba174606; no voters:
I20260810 06:48:36.141088 11011 raft_consensus.cc:2781] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:36.141494 11011 raft_consensus.cc:686] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 1 LEADER]: Becoming Leader. State: Replica: e13e424a6bb34e7cb816ebbfba174606, State: Running, Role: LEADER
I20260810 06:48:36.141919 11011 consensus_queue.cc:237] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } }
I20260810 06:48:36.146142 10222 catalog_manager.cc:5495] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 reported cstate change: term changed from 0 to 1, leader changed from <none> to e13e424a6bb34e7cb816ebbfba174606 (127.10.242.2). New cstate: current_term: 1 leader_uuid: "e13e424a6bb34e7cb816ebbfba174606" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e13e424a6bb34e7cb816ebbfba174606" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 36011 } health_report { overall_health: HEALTHY } } }
I20260810 06:48:36.196853 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:36.192049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.10.242.2:36011)
0810 06:48:36.192248 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:36.192254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:36.192298 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:48:36.192480 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:36.192484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:36.192560 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:36.192691 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:48:36.192990 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.193691 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.194983 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.195378 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.195463 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:36.195895 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:48:36.196360 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:48:36.196491 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:36.196549 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 06:48:36.196583 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 06:48:36.198549 10777 tablet.cc:2369] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:36.204991 10463 raft_consensus.cc:1261] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [term 1 FOLLOWER]: Refusing update from remote peer e13e424a6bb34e7cb816ebbfba174606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:36.205097 10895 raft_consensus.cc:1261] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [term 1 FOLLOWER]: Refusing update from remote peer e13e424a6bb34e7cb816ebbfba174606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:36.205914 11011 consensus_queue.cc:1035] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db7fcc0e77d74821bcc8a398df31edc2" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 34093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:36.206255 11016 consensus_queue.cc:1035] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2f35195cc634c6b835a1f0abeca479f" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:36.221316 11022 mvcc.cc:205] Tried to move back new op lower bound from 7316867138365763584 to 7316867138120982528. Current Snapshot: MvccSnapshot[applied={T|T < 7316867138365763584}]
I20260810 06:48:36.235364 10222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:36.235623 10222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:36.238965 10222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a23cc521c5ed40a5aa84d1d000b1eccc: Sending DeleteTablet for 3 replicas of tablet 04c2e5e4140c4e5caf543b262148d646
I20260810 06:48:36.240075 10866 tablet_service.cc:1514] Processing DeleteTablet for tablet 04c2e5e4140c4e5caf543b262148d646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:36 UTC) from {username='slave'} at 127.0.0.1:41626
I20260810 06:48:36.240185 10430 tablet_service.cc:1514] Processing DeleteTablet for tablet 04c2e5e4140c4e5caf543b262148d646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:36 UTC) from {username='slave'} at 127.0.0.1:56540
I20260810 06:48:36.241344 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet 04c2e5e4140c4e5caf543b262148d646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:36 UTC) from {username='slave'} at 127.0.0.1:49384
I20260810 06:48:36.242062 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 10333
I20260810 06:48:36.250094 11041 tablet_replica.cc:330] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: stopping tablet replica
I20260810 06:48:36.250556 11041 raft_consensus.cc:2229] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:36.251063 11041 raft_consensus.cc:2258] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:36.251909 11042 tablet_replica.cc:330] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: stopping tablet replica
I20260810 06:48:36.252315 11042 raft_consensus.cc:2229] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:36.252779 11042 raft_consensus.cc:2258] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:36.255077 11041 ts_tablet_manager.cc:1905] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:36.255462 11042 ts_tablet_manager.cc:1905] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:36.262044 11041 ts_tablet_manager.cc:1918] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:36.262317 11041 log.cc:1196] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/04c2e5e4140c4e5caf543b262148d646
I20260810 06:48:36.262480 11042 ts_tablet_manager.cc:1918] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:36.262750 11042 log.cc:1196] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/04c2e5e4140c4e5caf543b262148d646
I20260810 06:48:36.262915 11041 ts_tablet_manager.cc:1939] T 04c2e5e4140c4e5caf543b262148d646 P db7fcc0e77d74821bcc8a398df31edc2: Deleting consensus metadata
I20260810 06:48:36.263342 11042 ts_tablet_manager.cc:1939] T 04c2e5e4140c4e5caf543b262148d646 P a2f35195cc634c6b835a1f0abeca479f: Deleting consensus metadata
I20260810 06:48:36.264902 10200 catalog_manager.cc:4841] TS db7fcc0e77d74821bcc8a398df31edc2 (127.10.242.3:34093): tablet 04c2e5e4140c4e5caf543b262148d646 (table test-table [id=68731ab4b9b54254911c86f2c278f448]) successfully deleted
I20260810 06:48:36.265013 10193 catalog_manager.cc:4841] TS a2f35195cc634c6b835a1f0abeca479f (127.10.242.1:43625): tablet 04c2e5e4140c4e5caf543b262148d646 (table test-table [id=68731ab4b9b54254911c86f2c278f448]) successfully deleted
I20260810 06:48:36.267371 11043 tablet_replica.cc:330] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: stopping tablet replica
I20260810 06:48:36.267850 11043 raft_consensus.cc:2229] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:48:36.268523 11043 raft_consensus.cc:2258] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:36.270588 11043 ts_tablet_manager.cc:1905] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:36.272825 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:36.243506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56550 (local address 127.10.242.1:43625)
0810 06:48:36.243706 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:36.243713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:36.266714 (+ 23001us) server_negotiation.cc:407] Connection header received
0810 06:48:36.266895 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:36.266900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:36.267007 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:36.267114 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:36.267622 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.268395 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.269958 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.270406 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.270518 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:36.271564 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:36.271572 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:36.271586 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:36.272118 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:36.272164 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:36.272185 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:36.272281 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:36.272417 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 06:48:36.272524 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:48:36.273613 10532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:36.279786 11043 ts_tablet_manager.cc:1918] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:36.279959 11043 log.cc:1196] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/04c2e5e4140c4e5caf543b262148d646
I20260810 06:48:36.280515 11043 ts_tablet_manager.cc:1939] T 04c2e5e4140c4e5caf543b262148d646 P e13e424a6bb34e7cb816ebbfba174606: Deleting consensus metadata
I20260810 06:48:36.282241 10193 catalog_manager.cc:4841] TS e13e424a6bb34e7cb816ebbfba174606 (127.10.242.2:36011): tablet 04c2e5e4140c4e5caf543b262148d646 (table test-table [id=68731ab4b9b54254911c86f2c278f448]) successfully deleted
I20260810 06:48:36.362814 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 10333
I20260810 06:48:36.389611 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 10545
I20260810 06:48:36.396574 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:36.390378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49406 (local address 127.10.242.2:36011)
0810 06:48:36.390618 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:36.390624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:36.390645 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:48:36.390808 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:36.390813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:36.390926 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:36.391019 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:36.391457 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.392234 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.393506 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.393949 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.394056 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:36.395184 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:36.395193 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:36.395208 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:36.395836 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:36.395885 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:36.395906 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:36.396006 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:36.396148 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 06:48:36.396245 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260810 06:48:36.397259 10766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:36.507085 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 10545
I20260810 06:48:36.534629 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 10779
I20260810 06:48:36.540951 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:36.535355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.10.242.3:34093)
0810 06:48:36.535514 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:36.535519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:36.535581 (+ 62us) server_negotiation.cc:407] Connection header received
0810 06:48:36.535726 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:36.535730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:36.535808 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:36.535889 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:36.536228 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.536926 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.538218 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:36.538654 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:36.538752 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:36.539724 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:36.539733 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:36.539749 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:36.540342 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:36.540390 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:36.540411 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:36.540494 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:36.540613 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 06:48:36.540702 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:48:36.541548 10964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:36.626871 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 10779
I20260810 06:48:36.652771 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 10166
I20260810 06:48:36.653898 10281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:36.761014 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 10166
2026-08-10T06:48:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2793 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:36 dist-test-slave-zj8q krb5kdc[11055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:36 dist-test-slave-zj8q krb5kdc[11055](info): set up 2 sockets
Aug 10 06:48:36 dist-test-slave-zj8q krb5kdc[11055](info): commencing operation
krb5kdc: starting...
W20260810 06:48:38.013583 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:38 dist-test-slave-zj8q krb5kdc[11055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:48:38.290400 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:45491
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:45491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:38.614746 11069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:38.614998 11069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:38.615105 11069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:38.622014 11069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:38.622115 11069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:38.622196 11069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:38.622264 11069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:38.622309 11069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:38.631641 11069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:45491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:45491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:38.632787 11069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:38.634016 11069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:38.644850 11078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:38.645010 11075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:38.644937 11081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:38.645908 11069 server_base.cc:1034] running on GCE node
I20260810 06:48:38.646709 11069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:38.648231 11069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:38.649439 11069 hybrid_clock.cc:648] HybridClock initialized: now 1786344518649386 us; error 37 us; skew 500 ppm
I20260810 06:48:38.649770 11069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:38 dist-test-slave-zj8q krb5kdc[11055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344518, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:38.652797 11069 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:48:38.654172 11069 webserver.cc:460] Webserver started at http://127.10.242.62:37839/ using document root <none> and password file <none>
I20260810 06:48:38.654834 11069 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:38.654953 11069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:38.655239 11069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:38.658128 11069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "0d844f655cf244b08760fa15c6c4b26a"
format_stamp: "Formatted at 2026-08-10 06:48:38 on dist-test-slave-zj8q"
server_key: "1a0e97d7fe1e99009a09f3b69b3f6a47"
server_key_iv: "9a879fbfd865679999fcbf8581641500"
server_key_version: "encryptionkey@0"
I20260810 06:48:38.658921 11069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "0d844f655cf244b08760fa15c6c4b26a"
format_stamp: "Formatted at 2026-08-10 06:48:38 on dist-test-slave-zj8q"
server_key: "1a0e97d7fe1e99009a09f3b69b3f6a47"
server_key_iv: "9a879fbfd865679999fcbf8581641500"
server_key_version: "encryptionkey@0"
I20260810 06:48:38.663820 11069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:48:38.667052 11096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:38.668216 11069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:48:38.668365 11069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "0d844f655cf244b08760fa15c6c4b26a"
format_stamp: "Formatted at 2026-08-10 06:48:38 on dist-test-slave-zj8q"
server_key: "1a0e97d7fe1e99009a09f3b69b3f6a47"
server_key_iv: "9a879fbfd865679999fcbf8581641500"
server_key_version: "encryptionkey@0"
I20260810 06:48:38.668534 11069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:38.690016 11069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:38.693977 11069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:38.694278 11069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:38.710865 11069 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:45491
I20260810 06:48:38.710886 11182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:45491 every 8 connection(s)
I20260810 06:48:38.712157 11069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:38.713814 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 11069
I20260810 06:48:38.713971 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:38.714465 11208 external_mini_cluster.cc:1428] Setting key 3024bdfdd434b32ab023d99cb115406d
I20260810 06:48:38.716050 11183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:48:38 dist-test-slave-zj8q krb5kdc[11055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:38.729598 11183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a: Bootstrap starting.
I20260810 06:48:38.733302 11183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:38.734509 11183 log.cc:824] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:38.737644 11183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a: No bootstrap required, opened a new log
I20260810 06:48:38.738026 11189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:38.716133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.10.242.62:45491)
0810 06:48:38.716902 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:38.716916 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:38.716968 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:48:38.719009 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:38.719036 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:38.719510 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:38.719951 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:38.721251 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:38.723242 (+ 1991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:38.724519 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:38.725535 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:38.725727 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:38.730244 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:38.730255 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:38.730271 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:38.730318 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:38.733964 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:38.734599 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:38.734608 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:38.734615 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:38.734811 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:38.735217 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:38.735220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:38.735222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:38.735725 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:38.736004 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:38.736385 (+ 381us) server_negotiation.cc:299] Negotiation successful
0810 06:48:38.736778 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":309,"threads_started":1}
I20260810 06:48:38.745886 11183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d844f655cf244b08760fa15c6c4b26a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45491 } }
I20260810 06:48:38.746229 11183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:38.746310 11183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d844f655cf244b08760fa15c6c4b26a, State: Initialized, Role: FOLLOWER
I20260810 06:48:38.746814 11183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d844f655cf244b08760fa15c6c4b26a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45491 } }
I20260810 06:48:38.747017 11183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:38.747113 11183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:38.747233 11183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:38.749197 11183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d844f655cf244b08760fa15c6c4b26a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45491 } }
I20260810 06:48:38.749635 11183 leader_election.cc:302] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d844f655cf244b08760fa15c6c4b26a; no voters:
I20260810 06:48:38.750123 11183 leader_election.cc:288] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:38.750267 11195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:38.751390 11195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [term 1 LEADER]: Becoming Leader. State: Replica: 0d844f655cf244b08760fa15c6c4b26a, State: Running, Role: LEADER
I20260810 06:48:38.751878 11195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d844f655cf244b08760fa15c6c4b26a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45491 } }
I20260810 06:48:38.752100 11183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:38.754820 11196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d844f655cf244b08760fa15c6c4b26a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d844f655cf244b08760fa15c6c4b26a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45491 } } }
I20260810 06:48:38.755137 11196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:38.754444 11197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d844f655cf244b08760fa15c6c4b26a. Latest consensus state: current_term: 1 leader_uuid: "0d844f655cf244b08760fa15c6c4b26a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d844f655cf244b08760fa15c6c4b26a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45491 } } }
I20260810 06:48:38.755581 11197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:38.758638 11207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:38.762883 11207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:38.772418 11207 catalog_manager.cc:1347] Generated new cluster ID: 181702a577b94c1a98db7b3e6468f1ff
I20260810 06:48:38.772527 11207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:38.783120 11207 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:38.784335 11207 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:38.795408 11207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a: Generated new TSK 0
I20260810 06:48:38.796176 11207 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.242.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
Entry for principal kudu/127.10.242.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab.
I20260810 06:48:38.870874 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45491
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:39.205907 11231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:39.206180 11231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:39.206326 11231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:39.213305 11231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:39.213407 11231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:39.213552 11231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:48:39.222806 11231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.1.keytab
--principal=kudu/127.10.242.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:39.223985 11231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:39.225333 11231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:39.238085 11238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:39.238199 11246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:39.238801 11231 server_base.cc:1034] running on GCE node
W20260810 06:48:39.239071 11237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:39.239699 11231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:39.240574 11231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:39.241776 11231 hybrid_clock.cc:648] HybridClock initialized: now 1786344519241722 us; error 50 us; skew 500 ppm
I20260810 06:48:39.242050 11231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:39 dist-test-slave-zj8q krb5kdc[11055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344519, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:39.245328 11231 init.cc:377] Logged in from keytab as kudu/127.10.242.1@KRBTEST.COM (short username kudu)
I20260810 06:48:39.246701 11231 webserver.cc:460] Webserver started at http://127.10.242.1:44587/ using document root <none> and password file <none>
I20260810 06:48:39.247329 11231 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:39.247429 11231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:39.247689 11231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:39.250527 11231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "b7c09b4b375f477e92946b22ed041a11"
format_stamp: "Formatted at 2026-08-10 06:48:39 on dist-test-slave-zj8q"
server_key: "93ee0dc3b2fc76d26ff5294cd8092fba"
server_key_iv: "f53c8c9b29b4b9f32712cbbdaf1c8d8a"
server_key_version: "encryptionkey@0"
I20260810 06:48:39.251220 11231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "b7c09b4b375f477e92946b22ed041a11"
format_stamp: "Formatted at 2026-08-10 06:48:39 on dist-test-slave-zj8q"
server_key: "93ee0dc3b2fc76d26ff5294cd8092fba"
server_key_iv: "f53c8c9b29b4b9f32712cbbdaf1c8d8a"
server_key_version: "encryptionkey@0"
I20260810 06:48:39.256279 11231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:48:39.259472 11259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:39.260680 11231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:48:39.260826 11231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "b7c09b4b375f477e92946b22ed041a11"
format_stamp: "Formatted at 2026-08-10 06:48:39 on dist-test-slave-zj8q"
server_key: "93ee0dc3b2fc76d26ff5294cd8092fba"
server_key_iv: "f53c8c9b29b4b9f32712cbbdaf1c8d8a"
server_key_version: "encryptionkey@0"
I20260810 06:48:39.261016 11231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:39.280220 11231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:39.284113 11231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:39.284374 11231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:39.285426 11231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:39.286849 11231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:39.286927 11231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:39.287050 11231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:39.287107 11231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:39.316123 11231 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:37713
I20260810 06:48:39.316159 11426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:37713 every 8 connection(s)
I20260810 06:48:39.317386 11231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:39.318512 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 11231
I20260810 06:48:39.318698 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:48:39.319273 11208 external_mini_cluster.cc:1428] Setting key b9c427e998d65cf845df0366f2230590
Aug 10 06:48:39 dist-test-slave-zj8q krb5kdc[11055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344519, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.1@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:39.338548 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:39.320028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:39451 (local address 127.10.242.62:45491)
0810 06:48:39.320523 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:39.320530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:39.321555 (+ 1025us) server_negotiation.cc:407] Connection header received
0810 06:48:39.323958 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:39.323962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:39.324027 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:39.324124 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:39.325974 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:39.326755 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:39.328788 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:39.329206 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:39.329312 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:39.333119 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:39.333130 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:39.333133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:39.333165 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:39.335784 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:39.336437 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:39.336442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:39.336446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:39.336567 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:39.336927 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:39.336931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:39.336934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:39.337251 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:39.337387 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:39.337981 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 06:48:39.338218 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":236,"threads_started":1}
I20260810 06:48:39.339216 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:39.320623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:45491 (local address 127.10.242.1:39451)
0810 06:48:39.321373 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:39.321426 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:39.323659 (+ 2233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:39.324375 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:39.324386 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:39.325039 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:39.325786 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:39.325810 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:39.326915 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:39.326920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:39.328607 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:39.328622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:39.329431 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:39.329438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:39.329706 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:39.330400 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:39.330433 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:39.332908 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:39.335929 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:39.335935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:39.335946 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:39.336324 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:39.336648 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:39.336651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:39.336653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:39.336826 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:39.337356 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:39.337362 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:39.337630 (+ 268us) client_negotiation.cc:769] Sending connection context
0810 06:48:39.337851 (+ 221us) client_negotiation.cc:240] Negotiation successful
0810 06:48:39.338222 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":284,"threads_started":1}
I20260810 06:48:39.340583 11429 heartbeater.cc:347] Connected to a master server at 127.10.242.62:45491
I20260810 06:48:39.340896 11429 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:39.341797 11429 heartbeater.cc:511] Master 127.10.242.62:45491 requested a full tablet report, sending...
I20260810 06:48:39.343894 11125 ts_manager.cc:194] Registered new tserver with Master: b7c09b4b375f477e92946b22ed041a11 (127.10.242.1:37713)
I20260810 06:48:39.345283 11125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.1@KRBTEST.COM'} at 127.10.242.1:39451
WARNING: no policy specified for kudu/127.10.242.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
Entry for principal kudu/127.10.242.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab.
I20260810 06:48:39.383281 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45491
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:39.706385 11442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:39.706641 11442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:39.706771 11442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:39.715584 11442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:39.715704 11442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:39.715879 11442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:48:39.730540 11442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.2.keytab
--principal=kudu/127.10.242.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:39.731947 11442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:39.733363 11442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:39.745585 11448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:39.745683 11454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:39.745618 11450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:39.746639 11442 server_base.cc:1034] running on GCE node
I20260810 06:48:39.747254 11442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:39.748087 11442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:39.749281 11442 hybrid_clock.cc:648] HybridClock initialized: now 1786344519749232 us; error 41 us; skew 500 ppm
I20260810 06:48:39.749569 11442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:39 dist-test-slave-zj8q krb5kdc[11055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344519, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:39.752561 11442 init.cc:377] Logged in from keytab as kudu/127.10.242.2@KRBTEST.COM (short username kudu)
I20260810 06:48:39.753880 11442 webserver.cc:460] Webserver started at http://127.10.242.2:38611/ using document root <none> and password file <none>
I20260810 06:48:39.754555 11442 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:39.754673 11442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:39.754971 11442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:39.757766 11442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "aea42e3d76fd4f5caf33b79c55f99ca2"
format_stamp: "Formatted at 2026-08-10 06:48:39 on dist-test-slave-zj8q"
server_key: "fd493e29ce164d178516451eb8278904"
server_key_iv: "bae348a067d6a0616e397fa6e390c115"
server_key_version: "encryptionkey@0"
I20260810 06:48:39.758512 11442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "aea42e3d76fd4f5caf33b79c55f99ca2"
format_stamp: "Formatted at 2026-08-10 06:48:39 on dist-test-slave-zj8q"
server_key: "fd493e29ce164d178516451eb8278904"
server_key_iv: "bae348a067d6a0616e397fa6e390c115"
server_key_version: "encryptionkey@0"
I20260810 06:48:39.763338 11442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 06:48:39.766654 11462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:39.767889 11442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 06:48:39.768042 11442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "aea42e3d76fd4f5caf33b79c55f99ca2"
format_stamp: "Formatted at 2026-08-10 06:48:39 on dist-test-slave-zj8q"
server_key: "fd493e29ce164d178516451eb8278904"
server_key_iv: "bae348a067d6a0616e397fa6e390c115"
server_key_version: "encryptionkey@0"
I20260810 06:48:39.768214 11442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:39.792773 11442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:39.796922 11442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:39.797184 11442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:39.798175 11442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:39.799427 11442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:39.799502 11442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:39.799619 11442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:39.799672 11442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:39.843597 11442 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:38265
I20260810 06:48:39.843703 11648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:38265 every 8 connection(s)
I20260810 06:48:39.844952 11442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:48:39.850390 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 11442
I20260810 06:48:39.850557 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:48:39.850983 11208 external_mini_cluster.cc:1428] Setting key d7631403e43c673daf3c6f34920da32e
Aug 10 06:48:39 dist-test-slave-zj8q krb5kdc[11055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344519, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.2@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:39.871754 11654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:39.849785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:45491 (local address 127.10.242.2:38615)
0810 06:48:39.850679 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:39.850736 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:39.853511 (+ 2775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:39.854348 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:39.854360 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:39.855068 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:39.855876 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:39.855900 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:39.857332 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:39.857337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:39.859046 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:39.859061 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:39.860129 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:39.860135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:39.860398 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:39.861137 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:39.861165 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:39.863832 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:39.867373 (+ 3541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:39.867382 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:39.867397 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:39.867928 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:39.868377 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:39.868382 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:39.868385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:39.868594 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:39.869251 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:39.869258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:39.869622 (+ 364us) client_negotiation.cc:769] Sending connection context
0810 06:48:39.869904 (+ 282us) client_negotiation.cc:240] Negotiation successful
0810 06:48:39.870397 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":2627,"threads_started":1}
I20260810 06:48:39.872977 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:39.849219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:38615 (local address 127.10.242.62:45491)
0810 06:48:39.849652 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:39.849656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:39.851069 (+ 1413us) server_negotiation.cc:407] Connection header received
0810 06:48:39.853822 (+ 2753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:39.853828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:39.853910 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:39.854022 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:39.856044 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:39.857175 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:39.859205 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:39.859883 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:39.860019 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:39.864082 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:39.864087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:39.864091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:39.864129 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:39.867219 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:39.868059 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:39.868065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:39.868069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:39.868231 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:39.868718 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:39.868723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:39.868725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:39.869035 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:39.869163 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:39.872394 (+ 3231us) server_negotiation.cc:299] Negotiation successful
0810 06:48:39.872656 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":209,"threads_started":1}
I20260810 06:48:39.874234 11649 heartbeater.cc:347] Connected to a master server at 127.10.242.62:45491
I20260810 06:48:39.874531 11649 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:39.875229 11649 heartbeater.cc:511] Master 127.10.242.62:45491 requested a full tablet report, sending...
I20260810 06:48:39.876670 11125 ts_manager.cc:194] Registered new tserver with Master: aea42e3d76fd4f5caf33b79c55f99ca2 (127.10.242.2:38265)
I20260810 06:48:39.877456 11125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.2@KRBTEST.COM'} at 127.10.242.2:38615
WARNING: no policy specified for kudu/127.10.242.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
Entry for principal kudu/127.10.242.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab.
I20260810 06:48:39.914131 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45491
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:40.240332 11658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:40.240609 11658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:40.240751 11658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:40.247804 11658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:40.247906 11658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:40.248049 11658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:48:40.257469 11658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.3.keytab
--principal=kudu/127.10.242.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:45491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:40.258791 11658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:40.260036 11658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:40.272282 11675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:40.272351 11671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:40.272315 11669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:40.273485 11658 server_base.cc:1034] running on GCE node
I20260810 06:48:40.274082 11658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:40.274935 11658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:40.276134 11658 hybrid_clock.cc:648] HybridClock initialized: now 1786344520276084 us; error 37 us; skew 500 ppm
I20260810 06:48:40.276422 11658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:40 dist-test-slave-zj8q krb5kdc[11055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344520, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:40.282821 11658 init.cc:377] Logged in from keytab as kudu/127.10.242.3@KRBTEST.COM (short username kudu)
I20260810 06:48:40.284160 11658 webserver.cc:460] Webserver started at http://127.10.242.3:33691/ using document root <none> and password file <none>
I20260810 06:48:40.284777 11658 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:40.284893 11658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:40.285266 11658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:40.288094 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "8f45e0afe7164d1db0498d5b1c745dc4"
format_stamp: "Formatted at 2026-08-10 06:48:40 on dist-test-slave-zj8q"
server_key: "2049559350f963ff3b391196c487c7cf"
server_key_iv: "dba1ee3053825f559d83ed7a0eb6213c"
server_key_version: "encryptionkey@0"
I20260810 06:48:40.288816 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "8f45e0afe7164d1db0498d5b1c745dc4"
format_stamp: "Formatted at 2026-08-10 06:48:40 on dist-test-slave-zj8q"
server_key: "2049559350f963ff3b391196c487c7cf"
server_key_iv: "dba1ee3053825f559d83ed7a0eb6213c"
server_key_version: "encryptionkey@0"
I20260810 06:48:40.293697 11658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260810 06:48:40.296983 11682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:40.298200 11658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:48:40.298343 11658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "8f45e0afe7164d1db0498d5b1c745dc4"
format_stamp: "Formatted at 2026-08-10 06:48:40 on dist-test-slave-zj8q"
server_key: "2049559350f963ff3b391196c487c7cf"
server_key_iv: "dba1ee3053825f559d83ed7a0eb6213c"
server_key_version: "encryptionkey@0"
I20260810 06:48:40.298516 11658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:40.313210 11658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:40.317652 11658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:40.317973 11658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:40.319259 11658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:40.320888 11658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:40.320981 11658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:40.321105 11658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:40.321184 11658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:40.348488 11429 heartbeater.cc:503] Master 127.10.242.62:45491 was elected leader, sending a full tablet report...
I20260810 06:48:40.350596 11658 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:36331
I20260810 06:48:40.350637 11862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:36331 every 8 connection(s)
I20260810 06:48:40.351761 11658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:48:40.358732 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 11658
I20260810 06:48:40.358899 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:48:40.359328 11208 external_mini_cluster.cc:1428] Setting key 0a637fb97ad349d511133bbceeadede5
Aug 10 06:48:40 dist-test-slave-zj8q krb5kdc[11055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344520, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.3@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:48:40.369956 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.354088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:59261 (local address 127.10.242.62:45491)
0810 06:48:40.354271 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.354275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.355473 (+ 1198us) server_negotiation.cc:407] Connection header received
0810 06:48:40.357210 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.357215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.357295 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.357382 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:40.358823 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.359634 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.361407 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.361812 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.361916 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.365095 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:40.365100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:40.365103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:40.365134 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:40.367469 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.368080 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.368083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.368085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.368199 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.368587 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.368591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.368593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.368884 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:40.369006 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.369561 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.369735 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:48:40.370781 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.354607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:45491 (local address 127.10.242.3:59261)
0810 06:48:40.355325 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.355366 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:40.356963 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:40.357611 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:40.357619 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:40.358081 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:40.358664 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.358682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.359785 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.359789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.361280 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.361292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.362019 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.362027 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.362289 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.362810 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:48:40.362830 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:40.364899 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:40.367590 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:40.367596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:40.367606 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:40.367991 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:40.368274 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:48:40.368278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:48:40.368280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:48:40.368451 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:48:40.368986 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:40.368991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:40.369272 (+ 281us) client_negotiation.cc:769] Sending connection context
0810 06:48:40.369474 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 06:48:40.369799 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":269,"threads_started":1}
I20260810 06:48:40.372035 11863 heartbeater.cc:347] Connected to a master server at 127.10.242.62:45491
I20260810 06:48:40.372328 11863 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:40.373195 11863 heartbeater.cc:511] Master 127.10.242.62:45491 requested a full tablet report, sending...
I20260810 06:48:40.374645 11125 ts_manager.cc:194] Registered new tserver with Master: 8f45e0afe7164d1db0498d5b1c745dc4 (127.10.242.3:36331)
I20260810 06:48:40.375370 11125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.242.3@KRBTEST.COM'} at 127.10.242.3:59261
I20260810 06:48:40.385795 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:48:40.401522 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.389616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54346 (local address 127.10.242.62:45491)
0810 06:48:40.389785 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.389789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.390236 (+ 447us) server_negotiation.cc:407] Connection header received
0810 06:48:40.390417 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.390421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.390479 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.390556 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:40.391951 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.392654 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.393873 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.394310 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.394415 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.396657 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:40.396662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:40.396665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:40.396701 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:40.398959 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.399574 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.399578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.399580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.399706 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.400090 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.400096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.400099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.400405 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:40.400640 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.400997 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.401201 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:48:40.405092 11125 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:48:40.411705 11125 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:48:40.433610 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.423151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:38265 (local address 127.0.0.1:35544)
0810 06:48:40.424148 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.424164 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:40.424315 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:40.425434 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:40.425439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:40.425474 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:40.426020 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.426033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.428309 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.428315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.430814 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.430832 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.432626 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.432632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.432835 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.432890 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 06:48:40.432997 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 06:48:40.433171 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":27,"spinlock_wait_cycles":12544,"thread_start_us":748,"threads_started":1}
I20260810 06:48:40.434198 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.423733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.10.242.2:38265)
0810 06:48:40.424854 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.424864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.424892 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:48:40.424989 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.424997 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.425214 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.426249 (+ 1035us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:40.426506 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.428135 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.430989 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.432415 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.433417 (+ 1002us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.433602 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.433684 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.433845 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":952,"thread_start_us":834,"threads_started":1}
I20260810 06:48:40.434362 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.422727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:36331 (local address 127.0.0.1:56224)
0810 06:48:40.424080 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.424131 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:40.424315 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:40.425333 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:40.425340 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:40.425370 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:40.425904 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.425917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.428101 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.428107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.431489 (+ 3382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.431505 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.433188 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.433195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.433357 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.433389 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:48:40.433478 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:48:40.433605 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":1,"spinlock_wait_cycles":1408,"thread_start_us":1164,"threads_started":1}
I20260810 06:48:40.435104 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.422966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.10.242.3:36331)
0810 06:48:40.423717 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.423726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.424490 (+ 764us) server_negotiation.cc:407] Connection header received
0810 06:48:40.424599 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.424606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.424857 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.425041 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:40.426210 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.427912 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.431688 (+ 3776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.432969 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.433082 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.433305 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.434634 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.434797 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":375,"threads_started":1}
I20260810 06:48:40.438215 11536 tablet_service.cc:1467] Processing CreateTablet for tablet 34803377d0e8427a9ee043ab8c638928 (DEFAULT_TABLE table=test-table [id=88b77db17846495db604791c97080b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:40.438704 11765 tablet_service.cc:1467] Processing CreateTablet for tablet 34803377d0e8427a9ee043ab8c638928 (DEFAULT_TABLE table=test-table [id=88b77db17846495db604791c97080b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:40.439754 11536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34803377d0e8427a9ee043ab8c638928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:40.440097 11765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34803377d0e8427a9ee043ab8c638928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:40.440385 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.424644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37713 (local address 127.0.0.1:37176)
0810 06:48:40.431699 (+ 7055us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.431716 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:40.431818 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:40.433655 (+ 1837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:40.433660 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:40.433684 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:40.434142 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.434182 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.436501 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.436506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.437736 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.437747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.439685 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.439692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.439857 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.439883 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:48:40.439960 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 06:48:40.440078 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6936,"thread_start_us":6779,"threads_started":1}
I20260810 06:48:40.440465 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.425216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.10.242.1:37713)
0810 06:48:40.432853 (+ 7637us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.432861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.432888 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:48:40.432976 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.432984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.433207 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.433386 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:40.434649 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.436375 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.438003 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.439487 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.439608 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.439814 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.440013 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.440167 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7462,"thread_start_us":7315,"threads_started":1}
I20260810 06:48:40.443427 11334 tablet_service.cc:1467] Processing CreateTablet for tablet 34803377d0e8427a9ee043ab8c638928 (DEFAULT_TABLE table=test-table [id=88b77db17846495db604791c97080b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:40.444449 11334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34803377d0e8427a9ee043ab8c638928. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:40.454386 11890 tablet_bootstrap.cc:492] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: Bootstrap starting.
I20260810 06:48:40.455228 11893 tablet_bootstrap.cc:492] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: Bootstrap starting.
I20260810 06:48:40.456213 11895 tablet_bootstrap.cc:492] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: Bootstrap starting.
I20260810 06:48:40.458487 11890 tablet_bootstrap.cc:654] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:40.459251 11893 tablet_bootstrap.cc:654] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:40.459401 11895 tablet_bootstrap.cc:654] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:40.459430 11890 log.cc:824] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:40.460469 11893 log.cc:824] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:40.460469 11895 log.cc:824] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:40.463200 11895 tablet_bootstrap.cc:492] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: No bootstrap required, opened a new log
I20260810 06:48:40.463433 11890 tablet_bootstrap.cc:492] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: No bootstrap required, opened a new log
I20260810 06:48:40.463465 11895 ts_tablet_manager.cc:1397] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260810 06:48:40.463678 11890 ts_tablet_manager.cc:1397] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260810 06:48:40.465207 11893 tablet_bootstrap.cc:492] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: No bootstrap required, opened a new log
I20260810 06:48:40.465452 11893 ts_tablet_manager.cc:1397] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260810 06:48:40.471026 11895 raft_consensus.cc:348] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } }
I20260810 06:48:40.471431 11895 raft_consensus.cc:374] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:40.471519 11895 raft_consensus.cc:729] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7c09b4b375f477e92946b22ed041a11, State: Initialized, Role: FOLLOWER
I20260810 06:48:40.472183 11895 consensus_queue.cc:260] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } }
I20260810 06:48:40.474078 11890 raft_consensus.cc:348] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } }
I20260810 06:48:40.474557 11890 raft_consensus.cc:374] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:40.474694 11890 raft_consensus.cc:729] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aea42e3d76fd4f5caf33b79c55f99ca2, State: Initialized, Role: FOLLOWER
I20260810 06:48:40.475376 11895 ts_tablet_manager.cc:1428] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: Time spent starting tablet: real 0.012s user 0.008s sys 0.002s
I20260810 06:48:40.475270 11890 consensus_queue.cc:260] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } }
I20260810 06:48:40.476320 11649 heartbeater.cc:503] Master 127.10.242.62:45491 was elected leader, sending a full tablet report...
I20260810 06:48:40.476810 11890 ts_tablet_manager.cc:1428] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 06:48:40.477252 11893 raft_consensus.cc:348] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } }
I20260810 06:48:40.477768 11893 raft_consensus.cc:374] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:40.477876 11893 raft_consensus.cc:729] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f45e0afe7164d1db0498d5b1c745dc4, State: Initialized, Role: FOLLOWER
I20260810 06:48:40.478461 11893 consensus_queue.cc:260] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } }
I20260810 06:48:40.481163 11863 heartbeater.cc:503] Master 127.10.242.62:45491 was elected leader, sending a full tablet report...
I20260810 06:48:40.481756 11893 ts_tablet_manager.cc:1428] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
W20260810 06:48:40.569712 11432 tablet.cc:2369] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:40.570024 11904 raft_consensus.cc:482] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:40.570374 11904 raft_consensus.cc:504] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } }
I20260810 06:48:40.572283 11904 leader_election.cc:288] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f45e0afe7164d1db0498d5b1c745dc4 (127.10.242.3:36331), aea42e3d76fd4f5caf33b79c55f99ca2 (127.10.242.2:38265)
I20260810 06:48:40.579067 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.572723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:60187 (local address 127.10.242.3:36331)
0810 06:48:40.573020 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.573027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.573793 (+ 766us) server_negotiation.cc:407] Connection header received
0810 06:48:40.573962 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.573966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.574026 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.574097 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:40.574710 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.575439 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.577373 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.578119 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.578299 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.578374 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.578695 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.578817 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260810 06:48:40.579078 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.572908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:47777 (local address 127.10.242.2:38265)
0810 06:48:40.573142 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.573148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.573714 (+ 566us) server_negotiation.cc:407] Connection header received
0810 06:48:40.573921 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.573925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.573984 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.574057 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:40.574767 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.575455 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.577379 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.578146 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.578318 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.578388 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.578725 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.578846 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:48:40.579180 11910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.572665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:36331 (local address 127.10.242.1:60187)
0810 06:48:40.573686 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.573701 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:40.573830 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:40.574127 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:40.574131 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:40.574174 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:40.574550 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.574563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.575567 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.575571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.577223 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.577236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.578353 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.578359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.578508 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.578561 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:48:40.578635 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:48:40.578751 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":91,"thread_start_us":729,"threads_started":1}
I20260810 06:48:40.579231 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.572577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:38265 (local address 127.10.242.1:47777)
0810 06:48:40.573612 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.573636 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:40.573786 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:40.574093 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:40.574097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:40.574129 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:40.574632 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.574643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.575576 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.575581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.577229 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:40.577242 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.578417 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.578422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:40.578575 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.578597 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:48:40.578694 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:48:40.578817 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":85,"thread_start_us":656,"threads_started":1}
I20260810 06:48:40.580355 11566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34803377d0e8427a9ee043ab8c638928" candidate_uuid: "b7c09b4b375f477e92946b22ed041a11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" is_pre_election: true
I20260810 06:48:40.580295 11794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34803377d0e8427a9ee043ab8c638928" candidate_uuid: "b7c09b4b375f477e92946b22ed041a11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" is_pre_election: true
I20260810 06:48:40.580830 11794 raft_consensus.cc:2454] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7c09b4b375f477e92946b22ed041a11 in term 0.
I20260810 06:48:40.580847 11566 raft_consensus.cc:2454] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7c09b4b375f477e92946b22ed041a11 in term 0.
I20260810 06:48:40.581480 11261 leader_election.cc:302] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f45e0afe7164d1db0498d5b1c745dc4, b7c09b4b375f477e92946b22ed041a11; no voters:
I20260810 06:48:40.581849 11904 raft_consensus.cc:2781] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:48:40.582002 11904 raft_consensus.cc:482] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:48:40.582113 11904 raft_consensus.cc:3037] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:40.584266 11904 raft_consensus.cc:504] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } }
I20260810 06:48:40.584844 11904 leader_election.cc:288] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [CANDIDATE]: Term 1 election: Requested vote from peers 8f45e0afe7164d1db0498d5b1c745dc4 (127.10.242.3:36331), aea42e3d76fd4f5caf33b79c55f99ca2 (127.10.242.2:38265)
I20260810 06:48:40.585282 11794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34803377d0e8427a9ee043ab8c638928" candidate_uuid: "b7c09b4b375f477e92946b22ed041a11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f45e0afe7164d1db0498d5b1c745dc4"
I20260810 06:48:40.585311 11566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34803377d0e8427a9ee043ab8c638928" candidate_uuid: "b7c09b4b375f477e92946b22ed041a11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2"
I20260810 06:48:40.585492 11794 raft_consensus.cc:3037] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:40.585492 11566 raft_consensus.cc:3037] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:40.588665 11794 raft_consensus.cc:2454] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7c09b4b375f477e92946b22ed041a11 in term 1.
I20260810 06:48:40.588677 11566 raft_consensus.cc:2454] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7c09b4b375f477e92946b22ed041a11 in term 1.
I20260810 06:48:40.589066 11261 leader_election.cc:302] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8f45e0afe7164d1db0498d5b1c745dc4, aea42e3d76fd4f5caf33b79c55f99ca2, b7c09b4b375f477e92946b22ed041a11; no voters:
I20260810 06:48:40.589382 11904 raft_consensus.cc:2781] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:40.589839 11904 raft_consensus.cc:686] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 1 LEADER]: Becoming Leader. State: Replica: b7c09b4b375f477e92946b22ed041a11, State: Running, Role: LEADER
I20260810 06:48:40.590319 11904 consensus_queue.cc:237] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } }
I20260810 06:48:40.594677 11124 catalog_manager.cc:5495] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7c09b4b375f477e92946b22ed041a11 (127.10.242.1). New cstate: current_term: 1 leader_uuid: "b7c09b4b375f477e92946b22ed041a11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7c09b4b375f477e92946b22ed041a11" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 37713 } health_report { overall_health: HEALTHY } } }
W20260810 06:48:40.596925 11652 tablet.cc:2369] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:48:40.603302 11864 tablet.cc:2369] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:40.629339 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.624022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37188 (local address 127.10.242.1:37713)
0810 06:48:40.624271 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.624277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.624296 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:48:40.624409 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.624413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.624490 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.624620 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:48:40.625010 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.625766 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.627195 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.627616 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.627744 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.628267 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:48:40.628782 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:48:40.628889 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.629008 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.629075 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 06:48:40.637692 11794 raft_consensus.cc:1261] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [term 1 FOLLOWER]: Refusing update from remote peer b7c09b4b375f477e92946b22ed041a11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:40.637691 11566 raft_consensus.cc:1261] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [term 1 FOLLOWER]: Refusing update from remote peer b7c09b4b375f477e92946b22ed041a11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:40.638741 11915 consensus_queue.cc:1035] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f45e0afe7164d1db0498d5b1c745dc4" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 36331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:40.639096 11904 consensus_queue.cc:1035] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aea42e3d76fd4f5caf33b79c55f99ca2" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:40.666317 11125 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:40.666572 11125 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:40.669754 11125 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d844f655cf244b08760fa15c6c4b26a: Sending DeleteTablet for 3 replicas of tablet 34803377d0e8427a9ee043ab8c638928
I20260810 06:48:40.670943 11536 tablet_service.cc:1514] Processing DeleteTablet for tablet 34803377d0e8427a9ee043ab8c638928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:40 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:35544
I20260810 06:48:40.671041 11765 tablet_service.cc:1514] Processing DeleteTablet for tablet 34803377d0e8427a9ee043ab8c638928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:40 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:56224
I20260810 06:48:40.671839 11937 tablet_replica.cc:330] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: stopping tablet replica
I20260810 06:48:40.671993 11334 tablet_service.cc:1514] Processing DeleteTablet for tablet 34803377d0e8427a9ee043ab8c638928 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:40 UTC) from {username='kudu', principal='kudu/127.10.242.62@KRBTEST.COM'} at 127.0.0.1:37176
I20260810 06:48:40.672420 11937 raft_consensus.cc:2229] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:40.672530 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 11231
I20260810 06:48:40.675071 11938 tablet_replica.cc:330] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: stopping tablet replica
I20260810 06:48:40.675487 11938 raft_consensus.cc:2229] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:40.676005 11938 raft_consensus.cc:2258] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:40.678114 11938 ts_tablet_manager.cc:1905] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:40.679077 11937 raft_consensus.cc:2258] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:40.682045 11937 ts_tablet_manager.cc:1905] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:40.685475 11938 ts_tablet_manager.cc:1918] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:40.685711 11938 log.cc:1196] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/34803377d0e8427a9ee043ab8c638928
I20260810 06:48:40.686396 11938 ts_tablet_manager.cc:1939] T 34803377d0e8427a9ee043ab8c638928 P 8f45e0afe7164d1db0498d5b1c745dc4: Deleting consensus metadata
I20260810 06:48:40.688225 11100 catalog_manager.cc:4841] TS 8f45e0afe7164d1db0498d5b1c745dc4 (127.10.242.3:36331): tablet 34803377d0e8427a9ee043ab8c638928 (table test-table [id=88b77db17846495db604791c97080b4c]) successfully deleted
I20260810 06:48:40.689749 11939 tablet_replica.cc:330] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: stopping tablet replica
I20260810 06:48:40.690192 11939 raft_consensus.cc:2229] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:48:40.690992 11939 raft_consensus.cc:2258] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:40.693230 11937 ts_tablet_manager.cc:1918] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:40.693374 11937 log.cc:1196] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/34803377d0e8427a9ee043ab8c638928
I20260810 06:48:40.693809 11937 ts_tablet_manager.cc:1939] T 34803377d0e8427a9ee043ab8c638928 P aea42e3d76fd4f5caf33b79c55f99ca2: Deleting consensus metadata
I20260810 06:48:40.694080 11939 ts_tablet_manager.cc:1905] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:40.695254 11102 catalog_manager.cc:4841] TS aea42e3d76fd4f5caf33b79c55f99ca2 (127.10.242.2:38265): tablet 34803377d0e8427a9ee043ab8c638928 (table test-table [id=88b77db17846495db604791c97080b4c]) successfully deleted
Aug 10 06:48:40 dist-test-slave-zj8q krb5kdc[11055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.1@KRBTEST.COM
I20260810 06:48:40.704252 11939 ts_tablet_manager.cc:1918] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:40.704334 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.673217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.10.242.1:37713)
0810 06:48:40.673400 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.673406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.685859 (+ 12453us) server_negotiation.cc:407] Connection header received
0810 06:48:40.686383 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.686389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.686476 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.686549 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:40.688349 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.689229 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.690717 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.691440 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.691545 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.698229 (+ 6684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:40.698243 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:40.698260 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:40.698299 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:40.701515 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.702234 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.702241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.702243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.702435 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.702808 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.702814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.702817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.703149 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:40.703424 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.703757 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.703986 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:48:40.704430 11939 log.cc:1196] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/34803377d0e8427a9ee043ab8c638928
I20260810 06:48:40.704990 11939 ts_tablet_manager.cc:1939] T 34803377d0e8427a9ee043ab8c638928 P b7c09b4b375f477e92946b22ed041a11: Deleting consensus metadata
I20260810 06:48:40.705224 11421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:40.811196 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 11231
W20260810 06:48:40.845916 11100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.242.1:37713 (error 108)
I20260810 06:48:40.846000 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 11442
I20260810 06:48:40.847846 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.847422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37713 (local address 127.0.0.1:37208)
0810 06:48:40.847569 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.847679 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 06:48:40.848251 11100 catalog_manager.cc:4584] TS b7c09b4b375f477e92946b22ed041a11 (127.10.242.1:37713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 34803377d0e8427a9ee043ab8c638928: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37713: connect: Connection refused (error 111)
Aug 10 06:48:40 dist-test-slave-zj8q krb5kdc[11055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.2@KRBTEST.COM
I20260810 06:48:40.863603 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.847109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.10.242.2:38265)
0810 06:48:40.847285 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.847290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.847310 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:48:40.847840 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.847847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.847933 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.848000 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:40.849833 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.851038 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.852374 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.853015 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.853127 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.857363 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:40.857375 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:40.857395 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:40.857432 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:40.860969 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.861589 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.861598 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.861602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.861750 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.862067 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.862072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.862076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.862568 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:40.862844 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.863029 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.863246 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:48:40.864355 11638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:40.901921 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.901393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37713 (local address 127.0.0.1:37222)
0810 06:48:40.901601 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.901707 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 06:48:40.903007 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.902622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37713 (local address 127.0.0.1:37232)
0810 06:48:40.902765 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.902849 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:48:40.952683 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 11442
I20260810 06:48:40.963425 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.962909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37713 (local address 127.0.0.1:37240)
0810 06:48:40.963131 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.963242 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 06:48:40.964537 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.964148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37713 (local address 127.0.0.1:37254)
0810 06:48:40.964306 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:40.964407 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:48:40.978987 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 11658
Aug 10 06:48:40 dist-test-slave-zj8q krb5kdc[11055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.3@KRBTEST.COM
I20260810 06:48:40.993865 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:40.979686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56238 (local address 127.10.242.3:36331)
0810 06:48:40.979879 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:40.979886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:40.979907 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:48:40.980085 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:40.980091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:40.980178 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:40.980261 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:40.981573 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.982489 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.983778 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:40.984183 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:40.984294 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:40.988540 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:40.988555 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:40.988573 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:40.988606 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:40.991195 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.991813 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.991821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.991824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.991954 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:40.992340 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:40.992344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:40.992346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:40.992734 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:40.993032 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:40.993331 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 06:48:40.993546 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:48:40.994519 11856 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:41.045301 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:41.044715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37713 (local address 127.0.0.1:37260)
0810 06:48:41.044960 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:41.045079 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 06:48:41.046487 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:41.046005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37713 (local address 127.0.0.1:37272)
0810 06:48:41.046267 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:41.046356 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260810 06:48:41.081815 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 11658
I20260810 06:48:41.108963 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 11069
I20260810 06:48:41.110069 11175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:41.221087 11888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:41.220541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:37713 (local address 127.0.0.1:37284)
0810 06:48:41.220750 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:41.220868 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.1:37713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:48:41.221091 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 11069
2026-08-10T06:48:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4461 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T06:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:41Z Disabled control of system clock
I20260810 06:48:41.313066 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:33821
--webserver_interface=127.10.242.62
--webserver_port=0
--builtin_ntp_servers=127.10.242.20:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:33821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:48:41.643034 11953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:41.643293 11953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:41.643414 11953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:41.650478 11953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:41.650588 11953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:41.650651 11953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:41.650743 11953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:41.650804 11953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:41.659720 11953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:33821
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:33821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:41.661123 11953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:41.662945 11953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:41.674003 11959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:41.674052 11961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:41.674077 11958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:41.675040 11953 server_base.cc:1034] running on GCE node
I20260810 06:48:41.675762 11953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:41.677016 11953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:41.678233 11953 hybrid_clock.cc:648] HybridClock initialized: now 1786344521678194 us; error 28 us; skew 500 ppm
I20260810 06:48:41.678555 11953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:41.680609 11953 webserver.cc:460] Webserver started at http://127.10.242.62:39897/ using document root <none> and password file <none>
I20260810 06:48:41.681241 11953 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:41.681335 11953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:41.681633 11953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:41.684465 11953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "11252397556d4d769e6aa20c26fe2dcb"
format_stamp: "Formatted at 2026-08-10 06:48:41 on dist-test-slave-zj8q"
server_key: "a7b976a1fa11f64aad9fc68344e4e95c"
server_key_iv: "959416024cd0d712fa755f2c75ec49fc"
server_key_version: "encryptionkey@0"
I20260810 06:48:41.685240 11953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "11252397556d4d769e6aa20c26fe2dcb"
format_stamp: "Formatted at 2026-08-10 06:48:41 on dist-test-slave-zj8q"
server_key: "a7b976a1fa11f64aad9fc68344e4e95c"
server_key_iv: "959416024cd0d712fa755f2c75ec49fc"
server_key_version: "encryptionkey@0"
I20260810 06:48:41.690028 11953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260810 06:48:41.693285 11977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:41.694547 11953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:48:41.694712 11953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "11252397556d4d769e6aa20c26fe2dcb"
format_stamp: "Formatted at 2026-08-10 06:48:41 on dist-test-slave-zj8q"
server_key: "a7b976a1fa11f64aad9fc68344e4e95c"
server_key_iv: "959416024cd0d712fa755f2c75ec49fc"
server_key_version: "encryptionkey@0"
I20260810 06:48:41.694903 11953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:41.714457 11953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:41.715245 11953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:41.715505 11953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:41.732375 11953 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:33821
I20260810 06:48:41.732396 12058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:33821 every 8 connection(s)
I20260810 06:48:41.733632 11953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:41.737203 12059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:41.738801 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 11953
I20260810 06:48:41.738953 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:41.739406 11208 external_mini_cluster.cc:1428] Setting key 8d935c8bd03bdc6087b5eca96ecec376
I20260810 06:48:41.751089 12059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb: Bootstrap starting.
I20260810 06:48:41.753621 12063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:41.741195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.10.242.62:33821)
0810 06:48:41.742293 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:41.742307 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:41.742358 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:48:41.744083 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:41.744113 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:41.744688 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:41.745091 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:41.745592 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:41.747508 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:41.748774 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:41.749767 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:41.749934 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:41.750976 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:41.750986 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:41.751003 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:41.751698 (+ 695us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:41.751789 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:41.751815 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:41.751934 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:41.752092 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:48:41.752398 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":334,"threads_started":1}
I20260810 06:48:41.754813 12059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:41.756110 12059 log.cc:824] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:41.758807 12059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb: No bootstrap required, opened a new log
I20260810 06:48:41.766088 12059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11252397556d4d769e6aa20c26fe2dcb" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33821 } }
I20260810 06:48:41.766500 12059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:41.766583 12059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11252397556d4d769e6aa20c26fe2dcb, State: Initialized, Role: FOLLOWER
I20260810 06:48:41.767058 12059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11252397556d4d769e6aa20c26fe2dcb" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33821 } }
I20260810 06:48:41.767264 12059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:41.767427 12059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:41.767583 12059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:41.769629 12059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11252397556d4d769e6aa20c26fe2dcb" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33821 } }
I20260810 06:48:41.770080 12059 leader_election.cc:302] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11252397556d4d769e6aa20c26fe2dcb; no voters:
I20260810 06:48:41.770639 12059 leader_election.cc:288] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:41.770759 12067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:41.771713 12067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [term 1 LEADER]: Becoming Leader. State: Replica: 11252397556d4d769e6aa20c26fe2dcb, State: Running, Role: LEADER
I20260810 06:48:41.772163 12067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11252397556d4d769e6aa20c26fe2dcb" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33821 } }
I20260810 06:48:41.772483 12059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:41.774744 12070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11252397556d4d769e6aa20c26fe2dcb. Latest consensus state: current_term: 1 leader_uuid: "11252397556d4d769e6aa20c26fe2dcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11252397556d4d769e6aa20c26fe2dcb" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33821 } } }
I20260810 06:48:41.775161 12070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:41.775470 12068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11252397556d4d769e6aa20c26fe2dcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11252397556d4d769e6aa20c26fe2dcb" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 33821 } } }
I20260810 06:48:41.775748 12068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:41.777179 12079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:41.782019 12079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:41.790652 12079 catalog_manager.cc:1347] Generated new cluster ID: e897ee34c4064a3d8b22549990ea551f
I20260810 06:48:41.790755 12079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:41.827329 12079 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:41.828570 12079 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:41.835115 12079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb: Generated new TSK 0
I20260810 06:48:41.835868 12079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:41.853289 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.1:0
--local_ip_for_outbound_sockets=127.10.242.1
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33821
--builtin_ntp_servers=127.10.242.20:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:48:42.212919 12096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:42.213176 12096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:42.213294 12096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:42.220367 12096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:42.220458 12096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:42.220594 12096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.1
I20260810 06:48:42.229256 12096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.242.1
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.1
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:42.230528 12096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:42.232396 12096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:42.244855 12104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:42.244887 12105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:42.245123 12109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:42.245669 12096 server_base.cc:1034] running on GCE node
I20260810 06:48:42.246287 12096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:42.247224 12096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:42.248427 12096 hybrid_clock.cc:648] HybridClock initialized: now 1786344522248383 us; error 32 us; skew 500 ppm
I20260810 06:48:42.248720 12096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:42.250825 12096 webserver.cc:460] Webserver started at http://127.10.242.1:35975/ using document root <none> and password file <none>
I20260810 06:48:42.251507 12096 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:42.251609 12096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:42.251909 12096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:42.254834 12096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/data/instance:
uuid: "cc3dd71586e44f90a425f9eff781dc70"
format_stamp: "Formatted at 2026-08-10 06:48:42 on dist-test-slave-zj8q"
server_key: "7e7ee31916246b89829be6a3de56dc1d"
server_key_iv: "d6a915412c63b1e9d26aba5a42fb2f64"
server_key_version: "encryptionkey@0"
I20260810 06:48:42.255604 12096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance:
uuid: "cc3dd71586e44f90a425f9eff781dc70"
format_stamp: "Formatted at 2026-08-10 06:48:42 on dist-test-slave-zj8q"
server_key: "7e7ee31916246b89829be6a3de56dc1d"
server_key_iv: "d6a915412c63b1e9d26aba5a42fb2f64"
server_key_version: "encryptionkey@0"
I20260810 06:48:42.260784 12096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:48:42.264029 12118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:42.265231 12096 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:48:42.265375 12096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/wal
uuid: "cc3dd71586e44f90a425f9eff781dc70"
format_stamp: "Formatted at 2026-08-10 06:48:42 on dist-test-slave-zj8q"
server_key: "7e7ee31916246b89829be6a3de56dc1d"
server_key_iv: "d6a915412c63b1e9d26aba5a42fb2f64"
server_key_version: "encryptionkey@0"
I20260810 06:48:42.265579 12096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:42.279729 12096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:42.280483 12096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:42.280752 12096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:42.281663 12096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:42.283123 12096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:42.283221 12096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:42.283331 12096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:42.283407 12096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:42.313189 12096 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.1:43347
I20260810 06:48:42.313241 12311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.1:43347 every 8 connection(s)
I20260810 06:48:42.314472 12096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/data/info.pb
I20260810 06:48:42.318287 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 12096
I20260810 06:48:42.318480 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/wal/instance
I20260810 06:48:42.318948 11208 external_mini_cluster.cc:1428] Setting key 5454c9333c0e41a3a8b1cc89f47cf637
I20260810 06:48:42.321727 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.2:0
--local_ip_for_outbound_sockets=127.10.242.2
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33821
--builtin_ntp_servers=127.10.242.20:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:48:42.326628 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:42.316886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.1:52309 (local address 127.10.242.62:33821)
0810 06:48:42.317318 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:42.317323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:42.318508 (+ 1185us) server_negotiation.cc:407] Connection header received
0810 06:48:42.320489 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:42.320493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:42.320559 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:42.320674 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:42.321747 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:42.322554 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:42.324050 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:42.324449 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:42.324558 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:42.325813 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:42.325818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:42.325821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:42.325837 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:42.325875 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:42.325887 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:42.325958 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:42.326195 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 06:48:42.326292 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":210,"threads_started":1}
I20260810 06:48:42.327538 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:42.317454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:33821 (local address 127.10.242.1:52309)
0810 06:48:42.318349 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:42.318402 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:42.320235 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:42.320920 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:42.320931 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:42.321035 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:42.321587 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:42.321603 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:42.322700 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:42.322706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:42.323914 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:42.323927 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:42.324657 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:42.324665 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:42.324895 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:42.325506 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:48:42.325509 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:42.325558 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:48:42.325629 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:42.325973 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:42.325985 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:42.325993 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:48:42.326142 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 06:48:42.326531 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":269,"threads_started":1}
I20260810 06:48:42.328971 12313 heartbeater.cc:347] Connected to a master server at 127.10.242.62:33821
I20260810 06:48:42.329248 12313 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:42.329952 12313 heartbeater.cc:511] Master 127.10.242.62:33821 requested a full tablet report, sending...
I20260810 06:48:42.332091 12001 ts_manager.cc:194] Registered new tserver with Master: cc3dd71586e44f90a425f9eff781dc70 (127.10.242.1:43347)
I20260810 06:48:42.333954 12001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.1:52309
W20260810 06:48:42.654516 12322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:42.654810 12322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:42.654937 12322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:42.662001 12322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:42.662098 12322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:42.662259 12322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.2
I20260810 06:48:42.671078 12322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.242.2
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.2
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:42.672348 12322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:42.674103 12322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:42.686646 12332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:42.686666 12331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:42.686878 12336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:42.687762 12322 server_base.cc:1034] running on GCE node
I20260810 06:48:42.688311 12322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:42.689165 12322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:42.690375 12322 hybrid_clock.cc:648] HybridClock initialized: now 1786344522690327 us; error 31 us; skew 500 ppm
I20260810 06:48:42.690654 12322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:42.692694 12322 webserver.cc:460] Webserver started at http://127.10.242.2:34383/ using document root <none> and password file <none>
I20260810 06:48:42.693349 12322 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:42.693439 12322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:42.693698 12322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:42.696723 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/data/instance:
uuid: "01e4e36b880b4af3b7bd3342d88cfe28"
format_stamp: "Formatted at 2026-08-10 06:48:42 on dist-test-slave-zj8q"
server_key: "632a61562a392d49413f90c8c9277693"
server_key_iv: "bf28cc05cf689691ce48e4a0b434c0c9"
server_key_version: "encryptionkey@0"
I20260810 06:48:42.697500 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance:
uuid: "01e4e36b880b4af3b7bd3342d88cfe28"
format_stamp: "Formatted at 2026-08-10 06:48:42 on dist-test-slave-zj8q"
server_key: "632a61562a392d49413f90c8c9277693"
server_key_iv: "bf28cc05cf689691ce48e4a0b434c0c9"
server_key_version: "encryptionkey@0"
I20260810 06:48:42.702481 12322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260810 06:48:42.705680 12347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:42.706912 12322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:48:42.707059 12322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/wal
uuid: "01e4e36b880b4af3b7bd3342d88cfe28"
format_stamp: "Formatted at 2026-08-10 06:48:42 on dist-test-slave-zj8q"
server_key: "632a61562a392d49413f90c8c9277693"
server_key_iv: "bf28cc05cf689691ce48e4a0b434c0c9"
server_key_version: "encryptionkey@0"
I20260810 06:48:42.707268 12322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:42.722376 12322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:42.723096 12322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:42.723379 12322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:42.724289 12322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:42.725652 12322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:42.725747 12322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:42.725859 12322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:42.725929 12322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:42.756678 12322 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.2:38561
I20260810 06:48:42.756721 12529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.2:38561 every 8 connection(s)
I20260810 06:48:42.757897 12322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/data/info.pb
I20260810 06:48:42.763430 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 12322
I20260810 06:48:42.763589 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/wal/instance
I20260810 06:48:42.764045 11208 external_mini_cluster.cc:1428] Setting key 49004b7c001307636b15bae2e30d5cb9
I20260810 06:48:42.766865 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.242.3:0
--local_ip_for_outbound_sockets=127.10.242.3
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33821
--builtin_ntp_servers=127.10.242.20:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:48:42.770296 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:42.760347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.2:35105 (local address 127.10.242.62:33821)
0810 06:48:42.760569 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:42.760576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:42.761902 (+ 1326us) server_negotiation.cc:407] Connection header received
0810 06:48:42.763738 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:42.763744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:42.763829 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:42.763916 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:42.765110 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:42.765993 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:42.767603 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:42.767988 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:42.768099 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:42.769438 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:42.769443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:42.769447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:42.769463 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:42.769517 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:42.769534 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:42.769616 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:42.769822 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 06:48:42.769919 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":74}
I20260810 06:48:42.771258 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:42.760887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:33821 (local address 127.10.242.2:35105)
0810 06:48:42.761760 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:42.761796 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:42.763424 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:42.764162 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:42.764172 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:42.764266 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:42.764944 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:42.764965 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:42.766139 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:42.766146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:42.767464 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:42.767477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:42.768223 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:42.768231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:42.768480 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:42.769138 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:48:42.769141 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:42.769181 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:48:42.769256 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:42.769616 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:42.769636 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:42.769644 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:48:42.769798 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 06:48:42.770211 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":273,"threads_started":1}
I20260810 06:48:42.772672 12531 heartbeater.cc:347] Connected to a master server at 127.10.242.62:33821
I20260810 06:48:42.772979 12531 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:42.773746 12531 heartbeater.cc:511] Master 127.10.242.62:33821 requested a full tablet report, sending...
I20260810 06:48:42.775182 12001 ts_manager.cc:194] Registered new tserver with Master: 01e4e36b880b4af3b7bd3342d88cfe28 (127.10.242.2:38561)
I20260810 06:48:42.776168 12001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.2:35105
W20260810 06:48:43.099365 12538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:43.099642 12538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:43.099767 12538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:43.106803 12538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:43.106910 12538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:43.107055 12538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.242.3
I20260810 06:48:43.115943 12538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.242.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.242.3
--webserver_port=0
--tserver_master_addrs=127.10.242.62:33821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.242.3
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:43.117210 12538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:43.118986 12538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:43.131618 12545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:43.131623 12549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:43.131560 12544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:43.132390 12538 server_base.cc:1034] running on GCE node
I20260810 06:48:43.133005 12538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:43.133913 12538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:43.135133 12538 hybrid_clock.cc:648] HybridClock initialized: now 1786344523135074 us; error 32 us; skew 500 ppm
I20260810 06:48:43.135447 12538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:48:43.137583 12538 webserver.cc:460] Webserver started at http://127.10.242.3:44597/ using document root <none> and password file <none>
I20260810 06:48:43.138302 12538 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:43.138410 12538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:43.138717 12538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:43.141523 12538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/data/instance:
uuid: "432cc1dd2c76465ab7a29eb460c8768d"
format_stamp: "Formatted at 2026-08-10 06:48:43 on dist-test-slave-zj8q"
server_key: "43e421c5a0fa9f4e2c2b034f82da68b1"
server_key_iv: "f1c3a0da812850fed8367f8295362619"
server_key_version: "encryptionkey@0"
I20260810 06:48:43.142345 12538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance:
uuid: "432cc1dd2c76465ab7a29eb460c8768d"
format_stamp: "Formatted at 2026-08-10 06:48:43 on dist-test-slave-zj8q"
server_key: "43e421c5a0fa9f4e2c2b034f82da68b1"
server_key_iv: "f1c3a0da812850fed8367f8295362619"
server_key_version: "encryptionkey@0"
I20260810 06:48:43.147925 12538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:48:43.151458 12561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:43.152766 12538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:48:43.152904 12538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/wal
uuid: "432cc1dd2c76465ab7a29eb460c8768d"
format_stamp: "Formatted at 2026-08-10 06:48:43 on dist-test-slave-zj8q"
server_key: "43e421c5a0fa9f4e2c2b034f82da68b1"
server_key_iv: "f1c3a0da812850fed8367f8295362619"
server_key_version: "encryptionkey@0"
I20260810 06:48:43.153082 12538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:43.185570 12538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:43.186462 12538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:43.186754 12538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:43.187734 12538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:48:43.189177 12538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:48:43.189270 12538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:43.189378 12538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:48:43.189447 12538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:43.219401 12538 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.3:37857
I20260810 06:48:43.219472 12735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.3:37857 every 8 connection(s)
I20260810 06:48:43.220691 12538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/data/info.pb
I20260810 06:48:43.226513 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 12538
I20260810 06:48:43.226675 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/wal/instance
I20260810 06:48:43.227110 11208 external_mini_cluster.cc:1428] Setting key 69ce0bef8ad0b56406012965a8f0429b
I20260810 06:48:43.232962 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.223111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:48831 (local address 127.10.242.62:33821)
0810 06:48:43.223286 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.223291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.224609 (+ 1318us) server_negotiation.cc:407] Connection header received
0810 06:48:43.226399 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.226405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.226486 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.226576 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:43.227723 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.228754 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.230753 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.231147 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.231247 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.232384 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:43.232388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:43.232391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:43.232408 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:43.232434 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:43.232444 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:43.232506 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.232668 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.232739 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:48:43.233860 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.223616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:33821 (local address 127.10.242.3:48831)
0810 06:48:43.224467 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:43.224508 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:43.226074 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:43.226837 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:43.226845 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:43.226930 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:48:43.227543 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.227570 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.228901 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.228904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.230612 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.230632 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.231317 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.231322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.231509 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.232133 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:48:43.232136 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:48:43.232173 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:48:43.232246 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:48:43.232512 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:48:43.232519 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:48:43.232523 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 06:48:43.232651 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 06:48:43.232921 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":250,"threads_started":1}
I20260810 06:48:43.235065 12736 heartbeater.cc:347] Connected to a master server at 127.10.242.62:33821
I20260810 06:48:43.235323 12736 heartbeater.cc:464] Registering TS with master...
I20260810 06:48:43.235996 12736 heartbeater.cc:511] Master 127.10.242.62:33821 requested a full tablet report, sending...
I20260810 06:48:43.237336 12001 ts_manager.cc:194] Registered new tserver with Master: 432cc1dd2c76465ab7a29eb460c8768d (127.10.242.3:37857)
I20260810 06:48:43.238116 12001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.242.3:48831
I20260810 06:48:43.242002 11208 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:48:43.251612 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.245919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.10.242.62:33821)
0810 06:48:43.246075 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.246080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.246574 (+ 494us) server_negotiation.cc:407] Connection header received
0810 06:48:43.246733 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.246739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.246818 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.246899 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:43.247384 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.248107 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.249478 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.249878 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.249987 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.250975 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:43.250980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:43.250983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:43.250999 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:43.251029 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:43.251042 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:43.251114 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.251252 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.251342 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:48:43.254949 12001 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:48:43.261593 12001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:48:43.295738 12761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.272984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44504 (local address 127.10.242.1:43347)
0810 06:48:43.282348 (+ 9364us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.282357 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.286112 (+ 3755us) server_negotiation.cc:407] Connection header received
0810 06:48:43.287433 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.287443 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.287730 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.287907 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:43.289189 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.290692 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.293308 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.294585 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.294705 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.294948 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.295234 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.295395 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9184,"thread_start_us":9240,"threads_started":1}
I20260810 06:48:43.295756 12760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.272816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:43347 (local address 127.0.0.1:44504)
0810 06:48:43.285953 (+ 13137us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:43.286002 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:43.286921 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:43.288418 (+ 1497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:43.288427 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:43.288461 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:43.288861 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.288877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.290843 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.290846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.293063 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.293083 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.294768 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.294775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.295009 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.295057 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:48:43.295171 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 06:48:43.295363 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12892,"mutex_wait_us":4,"thread_start_us":17487,"threads_started":1}
I20260810 06:48:43.297315 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.273492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50812 (local address 127.10.242.3:37857)
0810 06:48:43.288858 (+ 15366us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.288867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.288903 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:48:43.289006 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.289014 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.289248 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.289442 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:43.290409 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.291757 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.294277 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.295665 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.295808 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.296103 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.296812 (+ 709us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.296976 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15184,"thread_start_us":15027,"threads_started":1}
I20260810 06:48:43.298107 12762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.272954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.3:37857 (local address 127.0.0.1:50812)
0810 06:48:43.288078 (+ 15124us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:43.288097 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:43.288240 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:43.289739 (+ 1499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:43.289744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:43.289772 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:43.290184 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.290196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.291939 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.291945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.294071 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.294086 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.296264 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.296283 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.296443 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.296478 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:48:43.296566 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 06:48:43.297813 (+ 1247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14979,"thread_start_us":12608,"threads_started":1}
I20260810 06:48:43.299067 12758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.273345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:38561 (local address 127.0.0.1:37614)
0810 06:48:43.290547 (+ 17202us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:43.290563 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:43.290688 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:43.291235 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:43.291240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:43.291264 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:43.291686 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.291698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.293174 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.293179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.295071 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.295086 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.298287 (+ 3201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.298294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.298483 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.298517 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:48:43.298617 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:48:43.298735 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17091,"mutex_wait_us":29,"thread_start_us":14762,"threads_started":1}
I20260810 06:48:43.300391 12203 tablet_service.cc:1467] Processing CreateTablet for tablet b04a960fdf3f43e8aaf802a1745787a4 (DEFAULT_TABLE table=test-table [id=415f3ef952a64308981d45156f90b968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:43.299189 12764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.273860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.10.242.2:38561)
0810 06:48:43.287501 (+ 13641us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.287510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.290688 (+ 3178us) server_negotiation.cc:407] Connection header received
0810 06:48:43.290835 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.290842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.291041 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.291228 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:43.291872 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.293006 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.295292 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.296727 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.296833 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.297050 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.298686 (+ 1636us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.298849 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13463,"thread_start_us":13642,"threads_started":1}
I20260810 06:48:43.301924 12203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04a960fdf3f43e8aaf802a1745787a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:43.301892 12641 tablet_service.cc:1467] Processing CreateTablet for tablet b04a960fdf3f43e8aaf802a1745787a4 (DEFAULT_TABLE table=test-table [id=415f3ef952a64308981d45156f90b968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:43.303328 12641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04a960fdf3f43e8aaf802a1745787a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:43.304282 12430 tablet_service.cc:1467] Processing CreateTablet for tablet b04a960fdf3f43e8aaf802a1745787a4 (DEFAULT_TABLE table=test-table [id=415f3ef952a64308981d45156f90b968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:48:43.305263 12430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b04a960fdf3f43e8aaf802a1745787a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:43.320102 12767 tablet_bootstrap.cc:492] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: Bootstrap starting.
I20260810 06:48:43.326897 12766 tablet_bootstrap.cc:492] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: Bootstrap starting.
I20260810 06:48:43.326910 12770 tablet_bootstrap.cc:492] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: Bootstrap starting.
I20260810 06:48:43.327876 12767 tablet_bootstrap.cc:654] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:43.329216 12767 log.cc:824] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:43.330436 12770 tablet_bootstrap.cc:654] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:43.331708 12766 tablet_bootstrap.cc:654] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:43.331718 12770 log.cc:824] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:43.332806 12766 log.cc:824] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:43.333416 12767 tablet_bootstrap.cc:492] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: No bootstrap required, opened a new log
I20260810 06:48:43.333699 12767 ts_tablet_manager.cc:1397] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.002s
I20260810 06:48:43.334656 12770 tablet_bootstrap.cc:492] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: No bootstrap required, opened a new log
I20260810 06:48:43.334990 12770 ts_tablet_manager.cc:1397] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 06:48:43.335740 12766 tablet_bootstrap.cc:492] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: No bootstrap required, opened a new log
I20260810 06:48:43.335996 12766 ts_tablet_manager.cc:1397] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260810 06:48:43.336900 12313 heartbeater.cc:503] Master 127.10.242.62:33821 was elected leader, sending a full tablet report...
I20260810 06:48:43.342417 12767 raft_consensus.cc:348] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } }
I20260810 06:48:43.342746 12767 raft_consensus.cc:374] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:43.342873 12767 raft_consensus.cc:729] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc3dd71586e44f90a425f9eff781dc70, State: Initialized, Role: FOLLOWER
I20260810 06:48:43.343465 12767 consensus_queue.cc:260] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } }
I20260810 06:48:43.345074 12770 raft_consensus.cc:348] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } }
I20260810 06:48:43.345472 12770 raft_consensus.cc:374] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:43.345592 12770 raft_consensus.cc:729] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432cc1dd2c76465ab7a29eb460c8768d, State: Initialized, Role: FOLLOWER
I20260810 06:48:43.346197 12770 consensus_queue.cc:260] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } }
I20260810 06:48:43.346302 12766 raft_consensus.cc:348] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } }
I20260810 06:48:43.346758 12766 raft_consensus.cc:374] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:43.346921 12766 raft_consensus.cc:729] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e4e36b880b4af3b7bd3342d88cfe28, State: Initialized, Role: FOLLOWER
I20260810 06:48:43.347534 12767 ts_tablet_manager.cc:1428] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: Time spent starting tablet: real 0.014s user 0.003s sys 0.011s
I20260810 06:48:43.347610 12766 consensus_queue.cc:260] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } }
I20260810 06:48:43.348708 12531 heartbeater.cc:503] Master 127.10.242.62:33821 was elected leader, sending a full tablet report...
I20260810 06:48:43.349382 12766 ts_tablet_manager.cc:1428] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260810 06:48:43.350816 12736 heartbeater.cc:503] Master 127.10.242.62:33821 was elected leader, sending a full tablet report...
I20260810 06:48:43.351264 12770 ts_tablet_manager.cc:1428] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260810 06:48:43.437750 12777 raft_consensus.cc:482] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:48:43.438077 12777 raft_consensus.cc:504] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } }
I20260810 06:48:43.439764 12777 leader_election.cc:288] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc3dd71586e44f90a425f9eff781dc70 (127.10.242.1:43347), 01e4e36b880b4af3b7bd3342d88cfe28 (127.10.242.2:38561)
I20260810 06:48:43.446398 12764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.440358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:52207 (local address 127.10.242.2:38561)
0810 06:48:43.440559 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.440564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.440746 (+ 182us) server_negotiation.cc:407] Connection header received
0810 06:48:43.440986 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.440993 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.441086 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.441193 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:43.441653 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.442584 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.444573 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.445543 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.445669 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.445719 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.446018 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.446104 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:48:43.446455 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.440291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.2:38561 (local address 127.10.242.3:52207)
0810 06:48:43.440619 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:43.440638 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:43.440804 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:43.441267 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:43.441271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:43.441300 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:43.441484 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.441494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.442732 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.442738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.444418 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.444430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.445700 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.445707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.445872 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.445909 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:48:43.446003 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 06:48:43.446134 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":31}
I20260810 06:48:43.447086 12761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.440264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.242.3:55553 (local address 127.10.242.1:43347)
0810 06:48:43.440471 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.440477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.441168 (+ 691us) server_negotiation.cc:407] Connection header received
0810 06:48:43.441387 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.441392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.441468 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.441517 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:48:43.442285 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.443131 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.445163 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.446103 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.446300 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.446363 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.446725 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.446851 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:48:43.447139 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.440354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.1:43347 (local address 127.10.242.3:55553)
0810 06:48:43.441062 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 06:48:43.441079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:48:43.441207 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:48:43.441713 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:48:43.441718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:48:43.441742 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:48:43.442091 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.442103 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.443298 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.443302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.444993 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:48:43.445008 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.446319 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.446325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:48:43.446501 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.446533 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:48:43.446680 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 06:48:43.446799 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":36,"thread_start_us":395,"threads_started":1}
I20260810 06:48:43.447516 12460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a960fdf3f43e8aaf802a1745787a4" candidate_uuid: "432cc1dd2c76465ab7a29eb460c8768d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" is_pre_election: true
I20260810 06:48:43.448042 12460 raft_consensus.cc:2454] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 432cc1dd2c76465ab7a29eb460c8768d in term 0.
I20260810 06:48:43.448057 12241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a960fdf3f43e8aaf802a1745787a4" candidate_uuid: "432cc1dd2c76465ab7a29eb460c8768d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc3dd71586e44f90a425f9eff781dc70" is_pre_election: true
I20260810 06:48:43.448558 12241 raft_consensus.cc:2454] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 432cc1dd2c76465ab7a29eb460c8768d in term 0.
I20260810 06:48:43.448745 12563 leader_election.cc:302] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01e4e36b880b4af3b7bd3342d88cfe28, 432cc1dd2c76465ab7a29eb460c8768d; no voters:
I20260810 06:48:43.449141 12777 raft_consensus.cc:2781] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:48:43.449285 12777 raft_consensus.cc:482] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:48:43.449376 12777 raft_consensus.cc:3037] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:43.451520 12777 raft_consensus.cc:504] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } }
I20260810 06:48:43.452054 12777 leader_election.cc:288] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [CANDIDATE]: Term 1 election: Requested vote from peers cc3dd71586e44f90a425f9eff781dc70 (127.10.242.1:43347), 01e4e36b880b4af3b7bd3342d88cfe28 (127.10.242.2:38561)
I20260810 06:48:43.452430 12241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a960fdf3f43e8aaf802a1745787a4" candidate_uuid: "432cc1dd2c76465ab7a29eb460c8768d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc3dd71586e44f90a425f9eff781dc70"
I20260810 06:48:43.452467 12460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b04a960fdf3f43e8aaf802a1745787a4" candidate_uuid: "432cc1dd2c76465ab7a29eb460c8768d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01e4e36b880b4af3b7bd3342d88cfe28"
I20260810 06:48:43.452596 12241 raft_consensus.cc:3037] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:43.452642 12460 raft_consensus.cc:3037] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:43.455020 12241 raft_consensus.cc:2454] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 432cc1dd2c76465ab7a29eb460c8768d in term 1.
I20260810 06:48:43.455140 12460 raft_consensus.cc:2454] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 432cc1dd2c76465ab7a29eb460c8768d in term 1.
I20260810 06:48:43.455400 12562 leader_election.cc:302] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 432cc1dd2c76465ab7a29eb460c8768d, cc3dd71586e44f90a425f9eff781dc70; no voters:
I20260810 06:48:43.455663 12777 raft_consensus.cc:2781] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:43.456085 12777 raft_consensus.cc:686] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 1 LEADER]: Becoming Leader. State: Replica: 432cc1dd2c76465ab7a29eb460c8768d, State: Running, Role: LEADER
I20260810 06:48:43.456497 12777 consensus_queue.cc:237] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } }
I20260810 06:48:43.460660 11999 catalog_manager.cc:5495] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d reported cstate change: term changed from 0 to 1, leader changed from <none> to 432cc1dd2c76465ab7a29eb460c8768d (127.10.242.3). New cstate: current_term: 1 leader_uuid: "432cc1dd2c76465ab7a29eb460c8768d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "432cc1dd2c76465ab7a29eb460c8768d" member_type: VOTER last_known_addr { host: "127.10.242.3" port: 37857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 } health_report { overall_health: UNKNOWN } } }
W20260810 06:48:43.472216 12738 tablet.cc:2369] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:43.501853 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.496963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50818 (local address 127.10.242.3:37857)
0810 06:48:43.497140 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.497146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.497167 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:48:43.497353 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.497357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.497446 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.497564 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:48:43.497877 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.498658 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.499996 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.500408 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.500498 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.500911 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:48:43.501391 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:48:43.501496 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.501532 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.501590 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:48:43.509703 12241 raft_consensus.cc:1261] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [term 1 FOLLOWER]: Refusing update from remote peer 432cc1dd2c76465ab7a29eb460c8768d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:48:43.509696 12460 raft_consensus.cc:1261] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [term 1 FOLLOWER]: Refusing update from remote peer 432cc1dd2c76465ab7a29eb460c8768d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260810 06:48:43.509828 12534 tablet.cc:2369] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:48:43.510666 12781 consensus_queue.cc:1035] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc3dd71586e44f90a425f9eff781dc70" member_type: VOTER last_known_addr { host: "127.10.242.1" port: 43347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:43.511046 12777 consensus_queue.cc:1035] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [LEADER]: Connected to new peer: Peer: permanent_uuid: "01e4e36b880b4af3b7bd3342d88cfe28" member_type: VOTER last_known_addr { host: "127.10.242.2" port: 38561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:48:43.521662 12794 mvcc.cc:205] Tried to move back new op lower bound from 7316867168285315072 to 7316867168081444864. Current Snapshot: MvccSnapshot[applied={T|T < 7316867168285315072}]
I20260810 06:48:43.528741 12792 mvcc.cc:205] Tried to move back new op lower bound from 7316867168285315072 to 7316867168081444864. Current Snapshot: MvccSnapshot[applied={T|T < 7316867168285315072 or (T in {7316867168285315072})}]
I20260810 06:48:43.533715 11999 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:43.533962 11999 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:48:43.537315 11999 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11252397556d4d769e6aa20c26fe2dcb: Sending DeleteTablet for 3 replicas of tablet b04a960fdf3f43e8aaf802a1745787a4
I20260810 06:48:43.538340 12203 tablet_service.cc:1514] Processing DeleteTablet for tablet b04a960fdf3f43e8aaf802a1745787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:43 UTC) from {username='slave'} at 127.0.0.1:44504
I20260810 06:48:43.538403 12641 tablet_service.cc:1514] Processing DeleteTablet for tablet b04a960fdf3f43e8aaf802a1745787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:43 UTC) from {username='slave'} at 127.0.0.1:50812
I20260810 06:48:43.538611 12430 tablet_service.cc:1514] Processing DeleteTablet for tablet b04a960fdf3f43e8aaf802a1745787a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:48:43 UTC) from {username='slave'} at 127.0.0.1:37614
I20260810 06:48:43.540242 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 12096
I20260810 06:48:43.550644 12804 tablet_replica.cc:330] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: stopping tablet replica
I20260810 06:48:43.551172 12807 tablet_replica.cc:330] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: stopping tablet replica
I20260810 06:48:43.551203 12809 tablet_replica.cc:330] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: stopping tablet replica
I20260810 06:48:43.551654 12809 raft_consensus.cc:2229] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:43.551735 12807 raft_consensus.cc:2229] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:48:43.552251 12804 raft_consensus.cc:2229] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:48:43.552438 12807 raft_consensus.cc:2258] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:43.552994 12809 raft_consensus.cc:2258] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:43.553563 12804 raft_consensus.cc:2258] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:48:43.555475 12807 ts_tablet_manager.cc:1905] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:43.555636 12809 ts_tablet_manager.cc:1905] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:43.557231 12804 ts_tablet_manager.cc:1905] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:48:43.562508 12809 ts_tablet_manager.cc:1918] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:43.562716 12809 log.cc:1196] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-1/wal/wals/b04a960fdf3f43e8aaf802a1745787a4
I20260810 06:48:43.563324 12809 ts_tablet_manager.cc:1939] T b04a960fdf3f43e8aaf802a1745787a4 P 01e4e36b880b4af3b7bd3342d88cfe28: Deleting consensus metadata
I20260810 06:48:43.565297 11979 catalog_manager.cc:4841] TS 01e4e36b880b4af3b7bd3342d88cfe28 (127.10.242.2:38561): tablet b04a960fdf3f43e8aaf802a1745787a4 (table test-table [id=415f3ef952a64308981d45156f90b968]) successfully deleted
I20260810 06:48:43.565364 12804 ts_tablet_manager.cc:1918] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:43.565538 12804 log.cc:1196] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-0/wal/wals/b04a960fdf3f43e8aaf802a1745787a4
I20260810 06:48:43.566030 12804 ts_tablet_manager.cc:1939] T b04a960fdf3f43e8aaf802a1745787a4 P cc3dd71586e44f90a425f9eff781dc70: Deleting consensus metadata
I20260810 06:48:43.567282 12807 ts_tablet_manager.cc:1918] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:48:43.567430 12807 log.cc:1196] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: Deleting WAL directory at /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344206489568-11208-0/minicluster-data/ts-2/wal/wals/b04a960fdf3f43e8aaf802a1745787a4
I20260810 06:48:43.567826 11978 catalog_manager.cc:4841] TS cc3dd71586e44f90a425f9eff781dc70 (127.10.242.1:43347): tablet b04a960fdf3f43e8aaf802a1745787a4 (table test-table [id=415f3ef952a64308981d45156f90b968]) successfully deleted
I20260810 06:48:43.568317 12807 ts_tablet_manager.cc:1939] T b04a960fdf3f43e8aaf802a1745787a4 P 432cc1dd2c76465ab7a29eb460c8768d: Deleting consensus metadata
I20260810 06:48:43.569800 11980 catalog_manager.cc:4841] TS 432cc1dd2c76465ab7a29eb460c8768d (127.10.242.3:37857): tablet b04a960fdf3f43e8aaf802a1745787a4 (table test-table [id=415f3ef952a64308981d45156f90b968]) successfully deleted
I20260810 06:48:43.570143 12761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.541804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.10.242.1:43347)
0810 06:48:43.541980 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.541987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.556231 (+ 14244us) server_negotiation.cc:407] Connection header received
0810 06:48:43.556516 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.556521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.556616 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.556697 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:43.557361 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.558465 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.560258 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.560911 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.561003 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.568429 (+ 7426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:43.568442 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:43.568460 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:43.569242 (+ 782us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:43.569305 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:43.569331 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:43.569451 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.569697 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.569808 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:48:43.570870 12304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:43.657119 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 12096
I20260810 06:48:43.682687 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 12322
I20260810 06:48:43.690045 12764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.683493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37630 (local address 127.10.242.2:38561)
0810 06:48:43.683682 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.683688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.683710 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:48:43.683781 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.683787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.683902 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.684014 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:43.684331 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.685492 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.686768 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.687406 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.687534 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.688492 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:43.688505 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:43.688526 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:43.689318 (+ 792us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:43.689379 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:43.689405 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:43.689522 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.689581 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.689686 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:48:43.690783 12524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:43.792232 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 12322
I20260810 06:48:43.818531 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 12538
I20260810 06:48:43.826116 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:43.819303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.10.242.3:37857)
0810 06:48:43.819458 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:43.819463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:43.819515 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:48:43.819729 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:43.819733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:43.819823 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:43.819913 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:43.820406 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.821159 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.822980 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:43.823397 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:43.823488 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:43.824714 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:43.824723 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:43.824741 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:43.825354 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:43.825407 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:43.825427 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:43.825507 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:43.825694 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 06:48:43.825768 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:48:43.826879 12730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:43.945343 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 12538
I20260810 06:48:43.968541 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 11953
I20260810 06:48:43.969655 12050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:44.080346 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 11953
2026-08-10T06:48:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2860 ms)
[----------] 6 tests from EncryptionPolicyTest (22835 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:44 dist-test-slave-zj8q krb5kdc[12819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:44 dist-test-slave-zj8q krb5kdc[12819](info): set up 2 sockets
Aug 10 06:48:44 dist-test-slave-zj8q krb5kdc[12819](info): commencing operation
krb5kdc: starting...
W20260810 06:48:45.343956 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:45 dist-test-slave-zj8q krb5kdc[12819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:48:45.633275 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:45.959311 12839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:45.959606 12839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:45.959754 12839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:45.966777 12839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:45.966884 12839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:45.966962 12839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:45.967046 12839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:45.967103 12839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:45.976444 12839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36143
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:45.977648 12839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:45.978930 12839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:45.989967 12850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:45.990024 12844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:45.990049 12845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:45.990986 12839 server_base.cc:1034] running on GCE node
I20260810 06:48:45.991726 12839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:45.993073 12839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:45.994251 12839 hybrid_clock.cc:648] HybridClock initialized: now 1786344525994200 us; error 44 us; skew 500 ppm
I20260810 06:48:45.994635 12839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:45 dist-test-slave-zj8q krb5kdc[12819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344525, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:46.001092 12839 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:48:46.002501 12839 webserver.cc:460] Webserver started at http://127.0.0.1:39041/ using document root <none> and password file <none>
I20260810 06:48:46.003165 12839 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:46.003289 12839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:46.003628 12839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:46.006707 12839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "0bc99526147c47e2ba9edcbe7dcbd451"
format_stamp: "Formatted at 2026-08-10 06:48:45 on dist-test-slave-zj8q"
server_key: "7080afb24d1a13e96e87da3ea94fc0b7"
server_key_iv: "06ccd8a58a3af5adcd6de32ac1c56848"
server_key_version: "encryptionkey@0"
I20260810 06:48:46.007460 12839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "0bc99526147c47e2ba9edcbe7dcbd451"
format_stamp: "Formatted at 2026-08-10 06:48:45 on dist-test-slave-zj8q"
server_key: "7080afb24d1a13e96e87da3ea94fc0b7"
server_key_iv: "06ccd8a58a3af5adcd6de32ac1c56848"
server_key_version: "encryptionkey@0"
I20260810 06:48:46.012593 12839 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:48:46.016250 12862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:46.017887 12839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:48:46.018131 12839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "0bc99526147c47e2ba9edcbe7dcbd451"
format_stamp: "Formatted at 2026-08-10 06:48:45 on dist-test-slave-zj8q"
server_key: "7080afb24d1a13e96e87da3ea94fc0b7"
server_key_iv: "06ccd8a58a3af5adcd6de32ac1c56848"
server_key_version: "encryptionkey@0"
I20260810 06:48:46.018399 12839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:46.041527 12839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:46.045539 12839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:46.045781 12839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:46.063133 12839 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36143
I20260810 06:48:46.063174 12940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36143 every 8 connection(s)
I20260810 06:48:46.064430 12839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:46.068009 12941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:46.068573 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 12839
I20260810 06:48:46.068786 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:46.069625 11208 external_mini_cluster.cc:1428] Setting key 5aaa8598673039c344adf0148365ea9d
I20260810 06:48:46.081781 12941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451: Bootstrap starting.
Aug 10 06:48:46 dist-test-slave-zj8q krb5kdc[12819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:48:46.085475 12941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:46.086606 12941 log.cc:824] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:46.089570 12941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451: No bootstrap required, opened a new log
I20260810 06:48:46.093434 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:46.071637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49849 (local address 127.0.0.1:36143)
0810 06:48:46.072411 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:46.072423 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:46.072463 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:48:46.074353 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:46.074376 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:46.074747 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:46.075063 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:46.076869 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:46.078202 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:46.079973 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:46.080611 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:46.080727 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:46.085491 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:46.085501 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:46.085527 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:46.085567 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:46.088956 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:46.089731 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:46.089739 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:46.089747 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:46.089958 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:46.090423 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:46.090428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:46.090431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:46.091099 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:46.091407 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:46.091800 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 06:48:46.092122 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":344,"threads_started":1}
I20260810 06:48:46.097260 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:48:46.098678 12941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc99526147c47e2ba9edcbe7dcbd451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } }
I20260810 06:48:46.099068 12941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:46.099164 12941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc99526147c47e2ba9edcbe7dcbd451, State: Initialized, Role: FOLLOWER
I20260810 06:48:46.099743 12941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc99526147c47e2ba9edcbe7dcbd451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } }
I20260810 06:48:46.100020 12941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:46.100172 12941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:46.100342 12941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:46.103345 12941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc99526147c47e2ba9edcbe7dcbd451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } }
I20260810 06:48:46.103899 12941 leader_election.cc:302] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bc99526147c47e2ba9edcbe7dcbd451; no voters:
I20260810 06:48:46.104547 12941 leader_election.cc:288] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:46.104605 12957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:46.105706 12957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [term 1 LEADER]: Becoming Leader. State: Replica: 0bc99526147c47e2ba9edcbe7dcbd451, State: Running, Role: LEADER
I20260810 06:48:46.106261 12957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc99526147c47e2ba9edcbe7dcbd451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } }
I20260810 06:48:46.106825 12941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:46.108745 12959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bc99526147c47e2ba9edcbe7dcbd451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc99526147c47e2ba9edcbe7dcbd451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } } }
I20260810 06:48:46.109095 12959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:46.110394 12960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bc99526147c47e2ba9edcbe7dcbd451. Latest consensus state: current_term: 1 leader_uuid: "0bc99526147c47e2ba9edcbe7dcbd451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bc99526147c47e2ba9edcbe7dcbd451" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36143 } } }
I20260810 06:48:46.110678 12960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:46.119167 12963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:46.120419 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:46.102826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52927 (local address 127.0.0.1:36143)
0810 06:48:46.102995 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:46.103002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:46.103487 (+ 485us) server_negotiation.cc:407] Connection header received
0810 06:48:46.103673 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:46.103685 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:46.103740 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:46.103813 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:46.105789 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:46.107006 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:46.108842 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:46.109617 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:46.109718 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:46.113049 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:46.113056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:46.113062 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:46.113102 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:46.116779 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:46.117715 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:46.117720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:46.117722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:46.117878 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:46.118522 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:46.118527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:46.118530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:46.118931 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:46.119185 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:46.119884 (+ 699us) server_negotiation.cc:299] Negotiation successful
0810 06:48:46.120053 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:48:46.124526 12963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:46.133517 12963 catalog_manager.cc:1347] Generated new cluster ID: 5272866a16ad4d1a8f07456cf5b86b2c
I20260810 06:48:46.133653 12963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:46.151753 12963 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:46.152998 12963 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:46.159731 12963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bc99526147c47e2ba9edcbe7dcbd451: Generated new TSK 0
I20260810 06:48:46.160516 12963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:46.169220 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 12839
I20260810 06:48:46.170482 12933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:46.258572 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 12839
W20260810 06:48:46.281200 12816 connection.cc:535] client connection to 127.0.0.1:36143 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36143): Connection reset by peer (error 104)
2026-08-10T06:48:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2169 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:46 dist-test-slave-zj8q krb5kdc[12990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:46 dist-test-slave-zj8q krb5kdc[12990](info): set up 2 sockets
Aug 10 06:48:46 dist-test-slave-zj8q krb5kdc[12990](info): commencing operation
krb5kdc: starting...
W20260810 06:48:47.500345 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:47 dist-test-slave-zj8q krb5kdc[12990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:48:47.771288 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:48.106240 13009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:48.106549 13009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:48.106698 13009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:48.113713 13009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:48.113814 13009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:48.113893 13009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:48.113976 13009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:48.114034 13009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:48.123459 13009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:48.124667 13009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:48.125989 13009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:48.137635 13014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:48.137598 13016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:48.137565 13018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:48.138309 13009 server_base.cc:1034] running on GCE node
I20260810 06:48:48.139089 13009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:48.140496 13009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:48.141705 13009 hybrid_clock.cc:648] HybridClock initialized: now 1786344528141650 us; error 46 us; skew 500 ppm
I20260810 06:48:48.142025 13009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:48 dist-test-slave-zj8q krb5kdc[12990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:48.145066 13009 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:48:48.146502 13009 webserver.cc:460] Webserver started at http://127.0.0.1:34351/ using document root <none> and password file <none>
I20260810 06:48:48.147163 13009 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:48.147284 13009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:48.147573 13009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:48.150575 13009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "c54b566b39d6475599dbe180375dec64"
format_stamp: "Formatted at 2026-08-10 06:48:48 on dist-test-slave-zj8q"
server_key: "fdc383664964eb29eb81c897fe1758df"
server_key_iv: "b33df446cdae782c369639004ca6432e"
server_key_version: "encryptionkey@0"
I20260810 06:48:48.151340 13009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "c54b566b39d6475599dbe180375dec64"
format_stamp: "Formatted at 2026-08-10 06:48:48 on dist-test-slave-zj8q"
server_key: "fdc383664964eb29eb81c897fe1758df"
server_key_iv: "b33df446cdae782c369639004ca6432e"
server_key_version: "encryptionkey@0"
I20260810 06:48:48.156286 13009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260810 06:48:48.159670 13026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:48.161104 13009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:48:48.161286 13009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "c54b566b39d6475599dbe180375dec64"
format_stamp: "Formatted at 2026-08-10 06:48:48 on dist-test-slave-zj8q"
server_key: "fdc383664964eb29eb81c897fe1758df"
server_key_iv: "b33df446cdae782c369639004ca6432e"
server_key_version: "encryptionkey@0"
I20260810 06:48:48.161477 13009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:48.174638 13009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:48.178210 13009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:48.178459 13009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:48.194847 13009 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46101
I20260810 06:48:48.194873 13106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46101 every 8 connection(s)
I20260810 06:48:48.196091 13009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:48.196753 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13009
I20260810 06:48:48.196938 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:48.197463 11208 external_mini_cluster.cc:1428] Setting key d7e9a94c634ec103c1abe2bdd43d72f5
I20260810 06:48:48.200055 13107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:48.212601 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:48.199272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54583 (local address 127.0.0.1:46101)
0810 06:48:48.200165 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:48.200182 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:48.200504 (+ 322us) server_negotiation.cc:407] Connection header received
0810 06:48:48.202769 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:48.202797 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:48.203307 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:48.203719 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:48.204207 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:48.206251 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:48.207600 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:48.208597 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:48.208766 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:48.209957 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:48.209968 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:48.209985 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:48.210721 (+ 736us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:48.210815 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:48.210850 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:48.211018 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:48.211243 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:48:48.211413 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":372,"threads_started":1}
I20260810 06:48:48.213635 13107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64: Bootstrap starting.
I20260810 06:48:48.216413 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:48:48.217156 13107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:48.218343 13107 log.cc:824] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:48.221025 13107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64: No bootstrap required, opened a new log
I20260810 06:48:48.227906 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:48.221215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39105 (local address 127.0.0.1:46101)
0810 06:48:48.221381 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:48.221387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:48.221885 (+ 498us) server_negotiation.cc:407] Connection header received
0810 06:48:48.222022 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:48.222027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:48.222106 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:48.222243 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:48.222711 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:48.223856 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:48.225123 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:48.225866 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:48.225991 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:48.227063 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:48.227070 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:48.227075 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:48.227095 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:48.227145 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:48.227165 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:48.227278 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:48.227548 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 06:48:48.227602 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:48:48.230104 13107 raft_consensus.cc:348] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54b566b39d6475599dbe180375dec64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } }
I20260810 06:48:48.230476 13107 raft_consensus.cc:374] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:48.230571 13107 raft_consensus.cc:729] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c54b566b39d6475599dbe180375dec64, State: Initialized, Role: FOLLOWER
I20260810 06:48:48.231112 13107 consensus_queue.cc:260] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54b566b39d6475599dbe180375dec64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } }
I20260810 06:48:48.231328 13107 raft_consensus.cc:388] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:48.231456 13107 raft_consensus.cc:482] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:48.231606 13107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:48.233655 13107 raft_consensus.cc:504] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54b566b39d6475599dbe180375dec64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } }
I20260810 06:48:48.234117 13107 leader_election.cc:302] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c54b566b39d6475599dbe180375dec64; no voters:
I20260810 06:48:48.234694 13107 leader_election.cc:288] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:48.234859 13119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:48.235792 13119 raft_consensus.cc:686] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [term 1 LEADER]: Becoming Leader. State: Replica: c54b566b39d6475599dbe180375dec64, State: Running, Role: LEADER
I20260810 06:48:48.236330 13119 consensus_queue.cc:237] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54b566b39d6475599dbe180375dec64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } }
I20260810 06:48:48.236693 13107 sys_catalog.cc:564] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:48.239655 13122 sys_catalog.cc:455] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c54b566b39d6475599dbe180375dec64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54b566b39d6475599dbe180375dec64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } } }
I20260810 06:48:48.239733 13123 sys_catalog.cc:455] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c54b566b39d6475599dbe180375dec64. Latest consensus state: current_term: 1 leader_uuid: "c54b566b39d6475599dbe180375dec64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c54b566b39d6475599dbe180375dec64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } } }
I20260810 06:48:48.240164 13123 sys_catalog.cc:458] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:48.240171 13122 sys_catalog.cc:458] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:48.241062 13135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:48.246949 13135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:48.255136 13135 catalog_manager.cc:1347] Generated new cluster ID: e0464909d87d437880a425bfe03e42f7
I20260810 06:48:48.255252 13135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:48.264420 13135 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:48.265717 13135 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:48.276412 13135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c54b566b39d6475599dbe180375dec64: Generated new TSK 0
I20260810 06:48:48.277104 13135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:48.283648 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13009
W20260810 06:48:48.284749 13099 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54583
W20260810 06:48:48.285451 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13009: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:48:48.285619 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13009
W20260810 06:48:48.312121 12986 connection.cc:535] client connection to 127.0.0.1:46101 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46101): Connection reset by peer (error 104)
2026-08-10T06:48:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2030 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:48 dist-test-slave-zj8q krb5kdc[13153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:48 dist-test-slave-zj8q krb5kdc[13153](info): set up 2 sockets
Aug 10 06:48:48 dist-test-slave-zj8q krb5kdc[13153](info): commencing operation
krb5kdc: starting...
W20260810 06:48:49.530275 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.007s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:49 dist-test-slave-zj8q krb5kdc[13153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:48:49.822876 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:50.165653 13171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:50.165973 13171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:50.166123 13171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:50.174280 13171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:50.174393 13171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:50.174469 13171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:50.174551 13171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:50.174611 13171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:50.184314 13171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:50.185585 13171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:50.186968 13171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:50.198309 13183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:50.198383 13177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:50.198350 13179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:50.199162 13171 server_base.cc:1034] running on GCE node
I20260810 06:48:50.199849 13171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:50.201105 13171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:50.202296 13171 hybrid_clock.cc:648] HybridClock initialized: now 1786344530202257 us; error 29 us; skew 500 ppm
I20260810 06:48:50.202613 13171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:50 dist-test-slave-zj8q krb5kdc[13153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344530, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:50.205610 13171 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:48:50.206977 13171 webserver.cc:460] Webserver started at http://127.0.0.1:33127/ using document root <none> and password file <none>
I20260810 06:48:50.207605 13171 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:50.207721 13171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:50.208012 13171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:50.210997 13171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "29c6fe44881b43b99ac414cb4ca96832"
format_stamp: "Formatted at 2026-08-10 06:48:50 on dist-test-slave-zj8q"
server_key: "878222ead606fe22fc0cbd9d5e747151"
server_key_iv: "3a64aeb5b30fdf6d48ddcd44d85e600b"
server_key_version: "encryptionkey@0"
I20260810 06:48:50.211779 13171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "29c6fe44881b43b99ac414cb4ca96832"
format_stamp: "Formatted at 2026-08-10 06:48:50 on dist-test-slave-zj8q"
server_key: "878222ead606fe22fc0cbd9d5e747151"
server_key_iv: "3a64aeb5b30fdf6d48ddcd44d85e600b"
server_key_version: "encryptionkey@0"
I20260810 06:48:50.216601 13171 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 06:48:50.220109 13191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:50.221334 13171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:48:50.221484 13171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "29c6fe44881b43b99ac414cb4ca96832"
format_stamp: "Formatted at 2026-08-10 06:48:50 on dist-test-slave-zj8q"
server_key: "878222ead606fe22fc0cbd9d5e747151"
server_key_iv: "3a64aeb5b30fdf6d48ddcd44d85e600b"
server_key_version: "encryptionkey@0"
I20260810 06:48:50.221655 13171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:50.227156 13171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:50.230883 13171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:50.231146 13171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:50.248108 13171 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33067
I20260810 06:48:50.248157 13284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33067 every 8 connection(s)
I20260810 06:48:50.249387 13171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:50.251121 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13171
I20260810 06:48:50.251303 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:50.251893 11208 external_mini_cluster.cc:1428] Setting key ada808c0fc2cd408d62697b7745e5b7b
I20260810 06:48:50.253597 13285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:50.260092 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:50.253549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36993 (local address 127.0.0.1:33067)
0810 06:48:50.254347 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:50.254361 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:50.254410 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:48:50.256426 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:50.256455 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:50.256875 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:50.257301 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:50.258263 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:50.258271 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:50.258287 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:50.258318 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:50.258407 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:50.258433 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:50.258631 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:50.258773 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 06:48:50.258951 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":303,"threads_started":1}
I20260810 06:48:50.263437 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:48:50.267385 13285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832: Bootstrap starting.
I20260810 06:48:50.270401 13285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:50.271627 13285 log.cc:824] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:50.272384 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:50.268488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59621 (local address 127.0.0.1:33067)
0810 06:48:50.268649 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:50.268655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:50.270641 (+ 1986us) server_negotiation.cc:407] Connection header received
0810 06:48:50.270777 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:50.270782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:50.270843 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:50.270945 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:50.271751 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:50.271757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:50.271761 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:50.271778 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:50.271821 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:50.271839 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:50.271949 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:50.272028 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:48:50.272082 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:48:50.274722 13285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832: No bootstrap required, opened a new log
I20260810 06:48:50.281854 13285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c6fe44881b43b99ac414cb4ca96832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33067 } }
I20260810 06:48:50.282256 13285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:50.282352 13285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c6fe44881b43b99ac414cb4ca96832, State: Initialized, Role: FOLLOWER
I20260810 06:48:50.282882 13285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c6fe44881b43b99ac414cb4ca96832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33067 } }
I20260810 06:48:50.283114 13285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:50.283243 13285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:50.283390 13285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:50.285910 13285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c6fe44881b43b99ac414cb4ca96832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33067 } }
I20260810 06:48:50.286432 13285 leader_election.cc:302] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29c6fe44881b43b99ac414cb4ca96832; no voters:
I20260810 06:48:50.286983 13285 leader_election.cc:288] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:50.287091 13296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:50.287775 13296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [term 1 LEADER]: Becoming Leader. State: Replica: 29c6fe44881b43b99ac414cb4ca96832, State: Running, Role: LEADER
I20260810 06:48:50.288290 13296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c6fe44881b43b99ac414cb4ca96832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33067 } }
I20260810 06:48:50.289147 13285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:50.292567 13298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29c6fe44881b43b99ac414cb4ca96832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c6fe44881b43b99ac414cb4ca96832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33067 } } }
I20260810 06:48:50.293020 13298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:50.297912 13300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29c6fe44881b43b99ac414cb4ca96832. Latest consensus state: current_term: 1 leader_uuid: "29c6fe44881b43b99ac414cb4ca96832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c6fe44881b43b99ac414cb4ca96832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33067 } } }
I20260810 06:48:50.298909 13300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:50.302654 13310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:50.307349 13310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:50.315349 13310 catalog_manager.cc:1347] Generated new cluster ID: 60808f02e65542a7a514236b53593ecc
I20260810 06:48:50.315449 13310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:50.330024 13310 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:50.330437 13310 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:50.340782 13310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29c6fe44881b43b99ac414cb4ca96832: Generated new TSK 0
I20260810 06:48:50.341506 13310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:50.353705 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13171
W20260810 06:48:50.354713 13279 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36993
W20260810 06:48:50.355335 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13171: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:48:50.355494 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13171
2026-08-10T06:48:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2071 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:50 dist-test-slave-zj8q krb5kdc[13331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:50 dist-test-slave-zj8q krb5kdc[13331](info): set up 2 sockets
Aug 10 06:48:50 dist-test-slave-zj8q krb5kdc[13331](info): commencing operation
krb5kdc: starting...
W20260810 06:48:51.592546 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:51 dist-test-slave-zj8q krb5kdc[13331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:48:51.874593 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42165
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:52.208665 13347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:52.208946 13347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:52.209075 13347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:52.216137 13347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:52.216239 13347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:52.216290 13347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:52.216349 13347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:52.216392 13347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:52.225800 13347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:52.226980 13347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:52.228204 13347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:52.239341 13356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:52.239423 13359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:52.239379 13361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:52.240376 13347 server_base.cc:1034] running on GCE node
I20260810 06:48:52.241096 13347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:52.242349 13347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:52.243582 13347 hybrid_clock.cc:648] HybridClock initialized: now 1786344532243506 us; error 38 us; skew 500 ppm
I20260810 06:48:52.243901 13347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:52 dist-test-slave-zj8q krb5kdc[13331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344532, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:52.250609 13347 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:48:52.251974 13347 webserver.cc:460] Webserver started at http://127.0.0.1:41987/ using document root <none> and password file <none>
I20260810 06:48:52.252578 13347 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:52.252698 13347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:52.252983 13347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:52.255826 13347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "af64fd0601df46589f27b85452675d26"
format_stamp: "Formatted at 2026-08-10 06:48:52 on dist-test-slave-zj8q"
server_key: "905ed4f31d2e1c06ea07fd55b28b3a2e"
server_key_iv: "fc07a9bbc29078cef7d4fcfbee6c5481"
server_key_version: "encryptionkey@0"
I20260810 06:48:52.256553 13347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "af64fd0601df46589f27b85452675d26"
format_stamp: "Formatted at 2026-08-10 06:48:52 on dist-test-slave-zj8q"
server_key: "905ed4f31d2e1c06ea07fd55b28b3a2e"
server_key_iv: "fc07a9bbc29078cef7d4fcfbee6c5481"
server_key_version: "encryptionkey@0"
I20260810 06:48:52.261340 13347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 06:48:52.264485 13372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:52.265651 13347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:48:52.265815 13347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "af64fd0601df46589f27b85452675d26"
format_stamp: "Formatted at 2026-08-10 06:48:52 on dist-test-slave-zj8q"
server_key: "905ed4f31d2e1c06ea07fd55b28b3a2e"
server_key_iv: "fc07a9bbc29078cef7d4fcfbee6c5481"
server_key_version: "encryptionkey@0"
I20260810 06:48:52.265982 13347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:52.288383 13347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:52.292168 13347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:52.292413 13347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:52.309659 13347 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42165
I20260810 06:48:52.309708 13455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42165 every 8 connection(s)
I20260810 06:48:52.311049 13347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:52.314664 13456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:52.321112 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13347
I20260810 06:48:52.321298 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:52.321918 11208 external_mini_cluster.cc:1428] Setting key ba74fed93704362cc02dd77f98a11004
I20260810 06:48:52.325379 13456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26: Bootstrap starting.
I20260810 06:48:52.328887 13456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:52.329927 13456 log.cc:824] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:52.332729 13456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26: No bootstrap required, opened a new log
Aug 10 06:48:52 dist-test-slave-zj8q krb5kdc[13331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:48:52.342348 13456 raft_consensus.cc:348] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af64fd0601df46589f27b85452675d26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42165 } }
I20260810 06:48:52.342783 13456 raft_consensus.cc:374] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:52.342895 13456 raft_consensus.cc:729] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af64fd0601df46589f27b85452675d26, State: Initialized, Role: FOLLOWER
I20260810 06:48:52.343519 13456 consensus_queue.cc:260] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af64fd0601df46589f27b85452675d26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42165 } }
I20260810 06:48:52.343806 13456 raft_consensus.cc:388] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:52.343955 13456 raft_consensus.cc:482] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:52.344187 13456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:52.345647 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:52.323763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54677 (local address 127.0.0.1:42165)
0810 06:48:52.324499 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:52.324510 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:52.324552 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:48:52.326321 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:52.326344 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:52.326715 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:52.327022 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:52.328831 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.330214 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.331733 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.332401 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.332539 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:52.337173 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:52.337187 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:52.337205 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:52.337259 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:52.341243 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:52.341885 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:52.341893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:52.341900 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:52.342097 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:52.342465 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:52.342469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:52.342472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:52.343167 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:52.343453 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:52.343874 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 06:48:52.344298 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":344,"threads_started":1}
I20260810 06:48:52.346995 13456 raft_consensus.cc:504] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af64fd0601df46589f27b85452675d26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42165 } }
I20260810 06:48:52.347460 13456 leader_election.cc:302] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af64fd0601df46589f27b85452675d26; no voters:
I20260810 06:48:52.348026 13456 leader_election.cc:288] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:52.348145 13462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:52.349216 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:48:52.349261 13462 raft_consensus.cc:686] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [term 1 LEADER]: Becoming Leader. State: Replica: af64fd0601df46589f27b85452675d26, State: Running, Role: LEADER
I20260810 06:48:52.349820 13462 consensus_queue.cc:237] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af64fd0601df46589f27b85452675d26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42165 } }
I20260810 06:48:52.350240 13456 sys_catalog.cc:564] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:52.352545 13467 sys_catalog.cc:455] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af64fd0601df46589f27b85452675d26. Latest consensus state: current_term: 1 leader_uuid: "af64fd0601df46589f27b85452675d26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af64fd0601df46589f27b85452675d26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42165 } } }
I20260810 06:48:52.352929 13467 sys_catalog.cc:458] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:52.356858 13465 sys_catalog.cc:455] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af64fd0601df46589f27b85452675d26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af64fd0601df46589f27b85452675d26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42165 } } }
I20260810 06:48:52.357230 13465 sys_catalog.cc:458] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:52.358546 13475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:52.363909 13475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:52.377573 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:52.359453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44563 (local address 127.0.0.1:42165)
0810 06:48:52.359627 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:52.359637 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:52.360239 (+ 602us) server_negotiation.cc:407] Connection header received
0810 06:48:52.360379 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:52.360384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:52.360465 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:52.360585 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:52.362636 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.363716 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.365578 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:52.366290 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:52.366407 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:52.369360 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:52.369367 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:52.369372 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:52.369414 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:52.373385 (+ 3971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:52.374312 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:52.374318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:52.374322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:52.374526 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:52.374990 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:52.374995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:52.374999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:52.375407 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:52.375652 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:52.376949 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0810 06:48:52.377190 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:48:52.383301 13475 catalog_manager.cc:1347] Generated new cluster ID: 1b86b80c9a99435783d5c4da422e419b
I20260810 06:48:52.383410 13475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:52.407274 13475 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:52.408573 13475 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:52.414070 13475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af64fd0601df46589f27b85452675d26: Generated new TSK 0
I20260810 06:48:52.414880 13475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:52.421347 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13347
I20260810 06:48:52.422479 13450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:52.510347 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13347
2026-08-10T06:48:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2150 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:52 dist-test-slave-zj8q krb5kdc[13506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:52 dist-test-slave-zj8q krb5kdc[13506](info): set up 2 sockets
Aug 10 06:48:52 dist-test-slave-zj8q krb5kdc[13506](info): commencing operation
krb5kdc: starting...
W20260810 06:48:53.749032 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:53 dist-test-slave-zj8q krb5kdc[13506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:48:54.026599 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:54.355139 13524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:54.355396 13524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:54.355532 13524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:54.362414 13524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:54.362509 13524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:54.362591 13524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:54.362671 13524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:54.362735 13524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:54.371966 13524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:54.373158 13524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:54.374415 13524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:54.385390 13532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:54.385596 13531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:54.385548 13534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:54.386492 13524 server_base.cc:1034] running on GCE node
I20260810 06:48:54.387219 13524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:54.388528 13524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:54.389724 13524 hybrid_clock.cc:648] HybridClock initialized: now 1786344534389689 us; error 46 us; skew 500 ppm
I20260810 06:48:54.390035 13524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:54 dist-test-slave-zj8q krb5kdc[13506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344534, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:54.392985 13524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:48:54.394387 13524 webserver.cc:460] Webserver started at http://127.0.0.1:42017/ using document root <none> and password file <none>
I20260810 06:48:54.394999 13524 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:54.395113 13524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:54.395388 13524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:54.398185 13524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "25a9d858239640e59fd5ee825ab09fb7"
format_stamp: "Formatted at 2026-08-10 06:48:54 on dist-test-slave-zj8q"
server_key: "6f2dab36228327aa65cdf6b1022be600"
server_key_iv: "7b04ebfe19573a1de899d4b3482add74"
server_key_version: "encryptionkey@0"
I20260810 06:48:54.398938 13524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "25a9d858239640e59fd5ee825ab09fb7"
format_stamp: "Formatted at 2026-08-10 06:48:54 on dist-test-slave-zj8q"
server_key: "6f2dab36228327aa65cdf6b1022be600"
server_key_iv: "7b04ebfe19573a1de899d4b3482add74"
server_key_version: "encryptionkey@0"
I20260810 06:48:54.403654 13524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:48:54.407056 13544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:54.408253 13524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:48:54.408418 13524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "25a9d858239640e59fd5ee825ab09fb7"
format_stamp: "Formatted at 2026-08-10 06:48:54 on dist-test-slave-zj8q"
server_key: "6f2dab36228327aa65cdf6b1022be600"
server_key_iv: "7b04ebfe19573a1de899d4b3482add74"
server_key_version: "encryptionkey@0"
I20260810 06:48:54.408587 13524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:54.421327 13524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:54.424813 13524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:54.425045 13524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:54.441802 13524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33555
I20260810 06:48:54.441836 13624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33555 every 8 connection(s)
I20260810 06:48:54.443019 13524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:54.446391 13628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:54.451619 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13524
I20260810 06:48:54.451823 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:54.452371 11208 external_mini_cluster.cc:1428] Setting key 4507811c08a90d804fe7dc9b2801cc2a
I20260810 06:48:54.457078 13628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7: Bootstrap starting.
I20260810 06:48:54.460487 13628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:54.461594 13628 log.cc:824] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:54.464476 13628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7: No bootstrap required, opened a new log
I20260810 06:48:54.465185 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:54.454088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41067 (local address 127.0.0.1:33555)
0810 06:48:54.454913 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:54.454924 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:54.454973 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:48:54.456605 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:54.456627 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:54.456974 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:54.457304 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:54.457871 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:54.459207 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:54.461028 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:54.461724 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:54.461860 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:54.463038 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:54.463046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:54.463059 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:54.463544 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:54.463615 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:54.463640 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:54.463735 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:54.463946 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 06:48:54.464160 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":368,"threads_started":1}
I20260810 06:48:54.468638 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:48:54.473450 13628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a9d858239640e59fd5ee825ab09fb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } }
I20260810 06:48:54.473899 13628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:54.473991 13628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a9d858239640e59fd5ee825ab09fb7, State: Initialized, Role: FOLLOWER
I20260810 06:48:54.474617 13628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a9d858239640e59fd5ee825ab09fb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } }
I20260810 06:48:54.474900 13628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:54.475028 13628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:54.475236 13628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:54.478012 13628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a9d858239640e59fd5ee825ab09fb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } }
I20260810 06:48:54.478597 13628 leader_election.cc:302] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25a9d858239640e59fd5ee825ab09fb7; no voters:
I20260810 06:48:54.479166 13628 leader_election.cc:288] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:54.479270 13644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:54.480607 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:54.473645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49587 (local address 127.0.0.1:33555)
0810 06:48:54.473815 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:54.473820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:54.474233 (+ 413us) server_negotiation.cc:407] Connection header received
0810 06:48:54.474444 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:54.474448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:54.474519 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:54.474618 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:54.475082 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:54.476123 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:54.477414 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:54.478089 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:54.478254 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:54.479394 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:54.479401 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:54.479406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:54.479429 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:54.479478 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:54.479503 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:54.479601 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:54.480151 (+ 550us) server_negotiation.cc:299] Negotiation successful
0810 06:48:54.480260 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:48:54.481056 13628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:54.481452 13644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 25a9d858239640e59fd5ee825ab09fb7, State: Running, Role: LEADER
I20260810 06:48:54.482007 13644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a9d858239640e59fd5ee825ab09fb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } }
I20260810 06:48:54.489017 13645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25a9d858239640e59fd5ee825ab09fb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a9d858239640e59fd5ee825ab09fb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } } }
I20260810 06:48:54.489078 13646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25a9d858239640e59fd5ee825ab09fb7. Latest consensus state: current_term: 1 leader_uuid: "25a9d858239640e59fd5ee825ab09fb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a9d858239640e59fd5ee825ab09fb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33555 } } }
I20260810 06:48:54.489403 13645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:54.489408 13646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:54.491334 13659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:54.495368 13659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:54.503216 13659 catalog_manager.cc:1347] Generated new cluster ID: c9e69574d00a4c58ac063acd2a8da3bd
I20260810 06:48:54.503312 13659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:54.509902 13659 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:54.511622 13659 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:54.521828 13659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25a9d858239640e59fd5ee825ab09fb7: Generated new TSK 0
I20260810 06:48:54.522656 13659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:54.535780 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13524
W20260810 06:48:54.536779 13617 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41067
W20260810 06:48:54.537393 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13524: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:48:54.537552 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13524
2026-08-10T06:48:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:54 dist-test-slave-zj8q krb5kdc[13672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:54 dist-test-slave-zj8q krb5kdc[13672](info): set up 2 sockets
Aug 10 06:48:54 dist-test-slave-zj8q krb5kdc[13672](info): commencing operation
krb5kdc: starting...
W20260810 06:48:55.780731 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:55 dist-test-slave-zj8q krb5kdc[13672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:48:56.068789 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:56.399948 13697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:56.400213 13697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:56.400315 13697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:56.407223 13697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:56.407325 13697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:56.407377 13697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:56.407434 13697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:56.407476 13697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:56.416796 13697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:56.417919 13697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:56.419202 13697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:56.430120 13708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:56.430174 13710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:56.430198 13712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:56.431246 13697 server_base.cc:1034] running on GCE node
I20260810 06:48:56.432011 13697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:56.433559 13697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:56.434758 13697 hybrid_clock.cc:648] HybridClock initialized: now 1786344536434721 us; error 30 us; skew 500 ppm
I20260810 06:48:56.435055 13697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:56 dist-test-slave-zj8q krb5kdc[13672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:56.437987 13697 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:48:56.439342 13697 webserver.cc:460] Webserver started at http://127.0.0.1:42605/ using document root <none> and password file <none>
I20260810 06:48:56.439929 13697 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:56.440025 13697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:56.440276 13697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:56.443138 13697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "6d707e7b64af4a8cbc1248d32a7d9292"
format_stamp: "Formatted at 2026-08-10 06:48:56 on dist-test-slave-zj8q"
server_key: "60b9b23873a8a03755999c3fb9879500"
server_key_iv: "cd49c7b835788dc99ba66cf56468ae31"
server_key_version: "encryptionkey@0"
I20260810 06:48:56.443846 13697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "6d707e7b64af4a8cbc1248d32a7d9292"
format_stamp: "Formatted at 2026-08-10 06:48:56 on dist-test-slave-zj8q"
server_key: "60b9b23873a8a03755999c3fb9879500"
server_key_iv: "cd49c7b835788dc99ba66cf56468ae31"
server_key_version: "encryptionkey@0"
I20260810 06:48:56.448614 13697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:48:56.451864 13728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:56.453032 13697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:48:56.453179 13697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "6d707e7b64af4a8cbc1248d32a7d9292"
format_stamp: "Formatted at 2026-08-10 06:48:56 on dist-test-slave-zj8q"
server_key: "60b9b23873a8a03755999c3fb9879500"
server_key_iv: "cd49c7b835788dc99ba66cf56468ae31"
server_key_version: "encryptionkey@0"
I20260810 06:48:56.453341 13697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:56.458523 13697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:56.461977 13697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:56.462291 13697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:56.478197 13697 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38531
I20260810 06:48:56.478219 13811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38531 every 8 connection(s)
I20260810 06:48:56.479417 13697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:56.482786 13812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:56.483583 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13697
I20260810 06:48:56.483822 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:56.484318 11208 external_mini_cluster.cc:1428] Setting key 4a93981259828a1d7fb3b61593adbf2a
I20260810 06:48:56.492487 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:56.486066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38749 (local address 127.0.0.1:38531)
0810 06:48:56.486889 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:56.486909 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:56.486963 (+ 54us) server_negotiation.cc:407] Connection header received
0810 06:48:56.488991 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:56.489020 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:56.489439 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:56.489866 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:56.490759 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:56.490766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:56.490782 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:56.490797 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:56.490855 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:56.490878 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:56.491075 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:56.491201 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 06:48:56.491373 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":312,"threads_started":1}
I20260810 06:48:56.495901 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:48:56.496310 13812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292: Bootstrap starting.
I20260810 06:48:56.500123 13812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:56.501327 13812 log.cc:824] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:56.503247 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:56.500770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44449 (local address 127.0.0.1:38531)
0810 06:48:56.500937 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:56.500943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:56.501403 (+ 460us) server_negotiation.cc:407] Connection header received
0810 06:48:56.501570 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:56.501574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:56.501620 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:56.501697 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:56.502666 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:56.502670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:56.502673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:48:56.502687 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:56.502723 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:48:56.502736 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:56.502813 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:56.502932 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 06:48:56.502971 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:48:56.504288 13812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292: No bootstrap required, opened a new log
I20260810 06:48:56.511281 13812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d707e7b64af4a8cbc1248d32a7d9292" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38531 } }
I20260810 06:48:56.511600 13812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:56.511691 13812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d707e7b64af4a8cbc1248d32a7d9292, State: Initialized, Role: FOLLOWER
I20260810 06:48:56.512284 13812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d707e7b64af4a8cbc1248d32a7d9292" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38531 } }
I20260810 06:48:56.512534 13812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:56.512660 13812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:56.512806 13812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:56.514875 13812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d707e7b64af4a8cbc1248d32a7d9292" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38531 } }
I20260810 06:48:56.515329 13812 leader_election.cc:302] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d707e7b64af4a8cbc1248d32a7d9292; no voters:
I20260810 06:48:56.515836 13812 leader_election.cc:288] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:56.515969 13827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:56.516825 13827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [term 1 LEADER]: Becoming Leader. State: Replica: 6d707e7b64af4a8cbc1248d32a7d9292, State: Running, Role: LEADER
I20260810 06:48:56.517364 13827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d707e7b64af4a8cbc1248d32a7d9292" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38531 } }
I20260810 06:48:56.517860 13812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:56.519995 13831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d707e7b64af4a8cbc1248d32a7d9292. Latest consensus state: current_term: 1 leader_uuid: "6d707e7b64af4a8cbc1248d32a7d9292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d707e7b64af4a8cbc1248d32a7d9292" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38531 } } }
I20260810 06:48:56.519985 13830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d707e7b64af4a8cbc1248d32a7d9292" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d707e7b64af4a8cbc1248d32a7d9292" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38531 } } }
I20260810 06:48:56.520449 13830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:56.520449 13831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:56.522872 13840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:56.527870 13840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:56.536581 13840 catalog_manager.cc:1347] Generated new cluster ID: af57408bc1194e0e9f38240f995437e4
I20260810 06:48:56.536682 13840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:56.553685 13840 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:56.554210 13840 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:56.562904 13840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d707e7b64af4a8cbc1248d32a7d9292: Generated new TSK 0
I20260810 06:48:56.563886 13840 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:56.571738 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13697
W20260810 06:48:56.572826 13802 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38749
W20260810 06:48:56.573444 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13697: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:48:56.573598 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13697
2026-08-10T06:48:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2032 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:56 dist-test-slave-zj8q krb5kdc[13868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:56 dist-test-slave-zj8q krb5kdc[13868](info): set up 2 sockets
Aug 10 06:48:56 dist-test-slave-zj8q krb5kdc[13868](info): commencing operation
krb5kdc: starting...
W20260810 06:48:57.812405 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:48:57 dist-test-slave-zj8q krb5kdc[13868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:48:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:48:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:48:58.096871 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44913
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:48:58.432590 13892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:48:58.432861 13892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:48:58.432999 13892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:48:58.439978 13892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:48:58.440079 13892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:48:58.440150 13892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:48:58.440225 13892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:48:58.440279 13892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:48:58.449797 13892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:48:58.451025 13892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:48:58.452286 13892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:48:58.463599 13899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:58.463526 13902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:48:58.463564 13906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:48:58.464711 13892 server_base.cc:1034] running on GCE node
I20260810 06:48:58.465498 13892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:48:58.467000 13892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:48:58.468219 13892 hybrid_clock.cc:648] HybridClock initialized: now 1786344538468176 us; error 28 us; skew 500 ppm
I20260810 06:48:58.468575 13892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:48:58 dist-test-slave-zj8q krb5kdc[13868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344538, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:48:58.475689 13892 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:48:58.477172 13892 webserver.cc:460] Webserver started at http://127.0.0.1:45077/ using document root <none> and password file <none>
I20260810 06:48:58.477837 13892 fs_manager.cc:359] Metadata directory not provided
I20260810 06:48:58.477958 13892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:48:58.478307 13892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:48:58.481192 13892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1"
format_stamp: "Formatted at 2026-08-10 06:48:58 on dist-test-slave-zj8q"
server_key: "85e0b8ec3893fc05aa222735fba648c2"
server_key_iv: "c171abe7eaf1203437800035150645e6"
server_key_version: "encryptionkey@0"
I20260810 06:48:58.481962 13892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1"
format_stamp: "Formatted at 2026-08-10 06:48:58 on dist-test-slave-zj8q"
server_key: "85e0b8ec3893fc05aa222735fba648c2"
server_key_iv: "c171abe7eaf1203437800035150645e6"
server_key_version: "encryptionkey@0"
I20260810 06:48:58.486940 13892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 06:48:58.490213 13917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:48:58.491443 13892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:48:58.491603 13892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1"
format_stamp: "Formatted at 2026-08-10 06:48:58 on dist-test-slave-zj8q"
server_key: "85e0b8ec3893fc05aa222735fba648c2"
server_key_iv: "c171abe7eaf1203437800035150645e6"
server_key_version: "encryptionkey@0"
I20260810 06:48:58.491793 13892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:48:58.527738 13892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:48:58.532032 13892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:48:58.532300 13892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:48:58.549964 13892 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44913
I20260810 06:48:58.549981 14001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44913 every 8 connection(s)
I20260810 06:48:58.551359 13892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:48:58.554572 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 13892
I20260810 06:48:58.554760 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:48:58.555325 14003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:48:58.555357 11208 external_mini_cluster.cc:1428] Setting key afca92c612b9d62f80080d1fd18c62e8
I20260810 06:48:58.567581 14003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1: Bootstrap starting.
Aug 10 06:48:58 dist-test-slave-zj8q krb5kdc[13868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:48:58.570721 14003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1: Neither blocks nor log segments found. Creating new log.
I20260810 06:48:58.572088 14003 log.cc:824] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:48:58.575230 14003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1: No bootstrap required, opened a new log
I20260810 06:48:58.578097 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:58.557176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:42595 (local address 127.0.0.1:44913)
0810 06:48:58.557929 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:58.557941 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:58.557980 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:48:58.560112 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:58.560133 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:58.560515 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:58.560895 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:58.562372 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:58.563649 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:58.564924 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:58.565649 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:58.565834 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:58.570027 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:58.570034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:58.570047 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:58.570086 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:58.573571 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:58.574374 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:58.574383 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:58.574391 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:58.574611 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:58.575042 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:58.575047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:58.575050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:58.575724 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:58.576070 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:58.576414 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 06:48:58.576742 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":326,"threads_started":1}
I20260810 06:48:58.581817 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:48:58.584661 14003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } }
I20260810 06:48:58.585105 14003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:48:58.585222 14003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d90755dc13b4ff8a9d7aabcd9ce97b1, State: Initialized, Role: FOLLOWER
I20260810 06:48:58.585877 14003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } }
I20260810 06:48:58.586201 14003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:48:58.586344 14003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:48:58.586522 14003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:48:58.589082 14003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } }
I20260810 06:48:58.589666 14003 leader_election.cc:302] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d90755dc13b4ff8a9d7aabcd9ce97b1; no voters:
I20260810 06:48:58.590334 14003 leader_election.cc:288] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:48:58.590422 14019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:48:58.592554 14003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:48:58.593571 14019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [term 1 LEADER]: Becoming Leader. State: Replica: 8d90755dc13b4ff8a9d7aabcd9ce97b1, State: Running, Role: LEADER
I20260810 06:48:58.594609 14019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } }
I20260810 06:48:58.599658 14022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d90755dc13b4ff8a9d7aabcd9ce97b1. Latest consensus state: current_term: 1 leader_uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } } }
I20260810 06:48:58.599736 14020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d90755dc13b4ff8a9d7aabcd9ce97b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44913 } } }
I20260810 06:48:58.600064 14022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:58.600071 14020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:48:58.605201 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:48:58.587185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:48067 (local address 127.0.0.1:44913)
0810 06:48:58.587364 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:48:58.587370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:48:58.587727 (+ 357us) server_negotiation.cc:407] Connection header received
0810 06:48:58.587853 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:48:58.587862 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:48:58.587939 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:48:58.588049 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:48:58.589441 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:58.590603 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:58.592391 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:48:58.593022 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:48:58.593172 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:48:58.596354 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:48:58.596360 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:48:58.596365 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:48:58.596403 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:48:58.600677 (+ 4274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:58.601698 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:58.601704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:58.601708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:58.601871 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:48:58.602532 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:48:58.602537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:48:58.602541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:48:58.602993 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:48:58.604437 (+ 1444us) server_negotiation.cc:1035] Waiting for connection context
0810 06:48:58.604618 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 06:48:58.604815 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 06:48:58.616837 14048 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:48:58.616956 14048 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:48:58.617080 14049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:48:58.618978 14049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:48:58.626780 14049 catalog_manager.cc:1347] Generated new cluster ID: 0d3336c4f2c84245805bde38235cf087
I20260810 06:48:58.626899 14049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:48:58.639564 14049 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:48:58.641350 14049 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:48:58.650812 14049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d90755dc13b4ff8a9d7aabcd9ce97b1: Generated new TSK 0
I20260810 06:48:58.651610 14049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:48:58.663416 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 13892
I20260810 06:48:58.664616 13993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:48:58.755247 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 13892
W20260810 06:48:58.780566 13865 connection.cc:535] client connection to 127.0.0.1:44913 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44913): Connection reset by peer (error 104)
2026-08-10T06:48:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2182 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:48:58 dist-test-slave-zj8q krb5kdc[14060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:48:58 dist-test-slave-zj8q krb5kdc[14060](info): set up 2 sockets
Aug 10 06:48:58 dist-test-slave-zj8q krb5kdc[14060](info): commencing operation
krb5kdc: starting...
W20260810 06:49:00.011379 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:00 dist-test-slave-zj8q krb5kdc[14060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:00.306893 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39711
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:00.636312 14086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:00.636600 14086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:00.636739 14086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:00.643724 14086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:00.643841 14086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:00.643919 14086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:00.644007 14086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:00.644062 14086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:00.653554 14086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:00.654870 14086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:00.656185 14086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:00.668017 14097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:00.668021 14093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:00.668260 14092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:00.669062 14086 server_base.cc:1034] running on GCE node
I20260810 06:49:00.669828 14086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:00.671211 14086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:00.672416 14086 hybrid_clock.cc:648] HybridClock initialized: now 1786344540672375 us; error 37 us; skew 500 ppm
I20260810 06:49:00.672736 14086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:00 dist-test-slave-zj8q krb5kdc[14060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344540, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:00.675812 14086 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:00.677254 14086 webserver.cc:460] Webserver started at http://127.0.0.1:44785/ using document root <none> and password file <none>
I20260810 06:49:00.677889 14086 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:00.678010 14086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:00.678364 14086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:00.681239 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "74bc097447a5476abca1fff244bda9c7"
format_stamp: "Formatted at 2026-08-10 06:49:00 on dist-test-slave-zj8q"
server_key: "ca46a024d07c276c5a333c15b8e4da2b"
server_key_iv: "1cd284e6804f1727f761b6906bcdc43e"
server_key_version: "encryptionkey@0"
I20260810 06:49:00.682016 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "74bc097447a5476abca1fff244bda9c7"
format_stamp: "Formatted at 2026-08-10 06:49:00 on dist-test-slave-zj8q"
server_key: "ca46a024d07c276c5a333c15b8e4da2b"
server_key_iv: "1cd284e6804f1727f761b6906bcdc43e"
server_key_version: "encryptionkey@0"
I20260810 06:49:00.686898 14086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 06:49:00.690119 14111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:00.691354 14086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:49:00.691506 14086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "74bc097447a5476abca1fff244bda9c7"
format_stamp: "Formatted at 2026-08-10 06:49:00 on dist-test-slave-zj8q"
server_key: "ca46a024d07c276c5a333c15b8e4da2b"
server_key_iv: "1cd284e6804f1727f761b6906bcdc43e"
server_key_version: "encryptionkey@0"
I20260810 06:49:00.691679 14086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:00.722023 14086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:00.726014 14086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:00.726318 14086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:00.744045 14086 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39711
I20260810 06:49:00.744081 14193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39711 every 8 connection(s)
I20260810 06:49:00.745402 14086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:00.749054 14194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:00.752756 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 14086
I20260810 06:49:00.752912 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:00.753325 11208 external_mini_cluster.cc:1428] Setting key e06c8a0efa560d467019163f92cef001
I20260810 06:49:00.760951 14194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7: Bootstrap starting.
I20260810 06:49:00.764390 14194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:00.765527 14194 log.cc:824] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:00.766079 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:00.754984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:55237 (local address 127.0.0.1:39711)
0810 06:49:00.755680 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:00.755691 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:00.755731 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:49:00.757298 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:00.757319 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:00.757705 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:00.758016 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:00.758649 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:00.759945 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:00.761824 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:00.762545 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:00.762671 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:00.763907 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:00.763915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:00.763928 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:00.764423 (+ 495us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:00.764500 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:00.764534 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:00.764659 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:00.764936 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 06:49:00.765098 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":308,"threads_started":1}
I20260810 06:49:00.768218 14194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7: No bootstrap required, opened a new log
I20260810 06:49:00.769588 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:00.777649 14194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bc097447a5476abca1fff244bda9c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39711 } }
I20260810 06:49:00.778092 14194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:00.778206 14194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74bc097447a5476abca1fff244bda9c7, State: Initialized, Role: FOLLOWER
I20260810 06:49:00.778800 14194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bc097447a5476abca1fff244bda9c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39711 } }
I20260810 06:49:00.779047 14194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:00.779184 14194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:00.779343 14194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:00.781073 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:00.774756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:41295 (local address 127.0.0.1:39711)
0810 06:49:00.774888 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:00.774892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:00.775229 (+ 337us) server_negotiation.cc:407] Connection header received
0810 06:49:00.775381 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:00.775384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:00.775446 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:00.775532 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:00.776088 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:00.776851 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:00.778605 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:00.779091 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:00.779206 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:00.780362 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:00.780370 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:00.780373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:00.780389 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:00.780445 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:00.780461 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:00.780559 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:00.780758 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:49:00.780807 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:49:00.781656 14194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bc097447a5476abca1fff244bda9c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39711 } }
I20260810 06:49:00.782124 14194 leader_election.cc:302] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74bc097447a5476abca1fff244bda9c7; no voters:
I20260810 06:49:00.782761 14194 leader_election.cc:288] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:00.782883 14212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:00.783648 14212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [term 1 LEADER]: Becoming Leader. State: Replica: 74bc097447a5476abca1fff244bda9c7, State: Running, Role: LEADER
I20260810 06:49:00.784148 14212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bc097447a5476abca1fff244bda9c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39711 } }
I20260810 06:49:00.784747 14194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:00.786918 14214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74bc097447a5476abca1fff244bda9c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bc097447a5476abca1fff244bda9c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39711 } } }
I20260810 06:49:00.787366 14214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:00.786947 14215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74bc097447a5476abca1fff244bda9c7. Latest consensus state: current_term: 1 leader_uuid: "74bc097447a5476abca1fff244bda9c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bc097447a5476abca1fff244bda9c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39711 } } }
I20260810 06:49:00.787861 14215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:00.788226 14222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:00.793020 14222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:00.804523 14222 catalog_manager.cc:1347] Generated new cluster ID: 8149831fdac74ccbbebea11caac29cac
I20260810 06:49:00.804636 14222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:00.848974 14222 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:00.850607 14222 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:00.859669 14222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74bc097447a5476abca1fff244bda9c7: Generated new TSK 0
I20260810 06:49:00.860431 14222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:00.865602 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14086
W20260810 06:49:00.867187 14185 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.236:55237
W20260810 06:49:00.867929 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14086: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:00.868067 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 14086
W20260810 06:49:00.891346 14055 connection.cc:535] client connection to 127.0.0.1:39711 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39711): Connection reset by peer (error 104)
2026-08-10T06:49:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2110 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:00 dist-test-slave-zj8q krb5kdc[14247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:00 dist-test-slave-zj8q krb5kdc[14247](info): set up 2 sockets
Aug 10 06:49:00 dist-test-slave-zj8q krb5kdc[14247](info): commencing operation
krb5kdc: starting...
W20260810 06:49:02.117740 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:02 dist-test-slave-zj8q krb5kdc[14247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:02.419381 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45305
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:02.751377 14262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:02.751659 14262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:02.751803 14262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:02.758940 14262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:02.759043 14262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:02.759122 14262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:02.759202 14262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:02.759260 14262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:02.768836 14262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:02.770108 14262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:02.771507 14262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:02.783280 14277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:02.783252 14271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:02.783475 14270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:02.784122 14262 server_base.cc:1034] running on GCE node
I20260810 06:49:02.784873 14262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:02.786274 14262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:02.787477 14262 hybrid_clock.cc:648] HybridClock initialized: now 1786344542787427 us; error 43 us; skew 500 ppm
I20260810 06:49:02.787802 14262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:02 dist-test-slave-zj8q krb5kdc[14247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344542, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:02.790882 14262 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:02.792202 14262 webserver.cc:460] Webserver started at http://127.0.0.1:42565/ using document root <none> and password file <none>
I20260810 06:49:02.792805 14262 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:02.792903 14262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:02.793146 14262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:02.796001 14262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "b9e08d1f51e9423b8e73ddf2093811d6"
format_stamp: "Formatted at 2026-08-10 06:49:02 on dist-test-slave-zj8q"
server_key: "26cfc5773e686530867a7e91f5ed0658"
server_key_iv: "ab97d91566b84128321d56377ed49981"
server_key_version: "encryptionkey@0"
I20260810 06:49:02.796725 14262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "b9e08d1f51e9423b8e73ddf2093811d6"
format_stamp: "Formatted at 2026-08-10 06:49:02 on dist-test-slave-zj8q"
server_key: "26cfc5773e686530867a7e91f5ed0658"
server_key_iv: "ab97d91566b84128321d56377ed49981"
server_key_version: "encryptionkey@0"
I20260810 06:49:02.801548 14262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:49:02.804844 14286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:02.806092 14262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:49:02.806288 14262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "b9e08d1f51e9423b8e73ddf2093811d6"
format_stamp: "Formatted at 2026-08-10 06:49:02 on dist-test-slave-zj8q"
server_key: "26cfc5773e686530867a7e91f5ed0658"
server_key_iv: "ab97d91566b84128321d56377ed49981"
server_key_version: "encryptionkey@0"
I20260810 06:49:02.806454 14262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:02.811672 14262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:02.815228 14262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:02.815531 14262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:02.832378 14262 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45305
I20260810 06:49:02.832422 14366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45305 every 8 connection(s)
I20260810 06:49:02.833721 14262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:02.835891 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 14262
I20260810 06:49:02.836036 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:02.836494 11208 external_mini_cluster.cc:1428] Setting key 0ce5ef5d14424f1aac5054bbdfc72c72
I20260810 06:49:02.837834 14367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:02.844798 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:02.838250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:39727 (local address 127.0.0.1:45305)
0810 06:49:02.839060 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:02.839075 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:02.839137 (+ 62us) server_negotiation.cc:407] Connection header received
0810 06:49:02.841025 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:02.841053 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:02.841494 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:02.841914 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:02.842896 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:02.842907 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:02.842924 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:02.842942 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:02.843038 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:02.843064 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:02.843286 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:02.843418 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 06:49:02.843616 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":319,"threads_started":1}
I20260810 06:49:02.848182 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:02.851948 14367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6: Bootstrap starting.
I20260810 06:49:02.855791 14367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:02.857334 14367 log.cc:824] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:02.859372 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:02.856072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:34141 (local address 127.0.0.1:45305)
0810 06:49:02.856230 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:02.856235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:02.856720 (+ 485us) server_negotiation.cc:407] Connection header received
0810 06:49:02.856897 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:02.856902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:02.856960 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:02.857065 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:02.858143 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:02.858170 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:02.858176 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:02.858197 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:02.858249 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:02.858269 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:02.858430 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:02.858563 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 06:49:02.858622 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:49:02.860782 14367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6: No bootstrap required, opened a new log
I20260810 06:49:02.867914 14367 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e08d1f51e9423b8e73ddf2093811d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260810 06:49:02.868230 14367 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:02.868300 14367 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9e08d1f51e9423b8e73ddf2093811d6, State: Initialized, Role: FOLLOWER
I20260810 06:49:02.868811 14367 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e08d1f51e9423b8e73ddf2093811d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260810 06:49:02.869007 14367 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:02.869114 14367 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:02.869233 14367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:02.871279 14367 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e08d1f51e9423b8e73ddf2093811d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260810 06:49:02.871711 14367 leader_election.cc:302] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9e08d1f51e9423b8e73ddf2093811d6; no voters:
I20260810 06:49:02.872206 14367 leader_election.cc:288] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:02.872318 14385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:02.872918 14385 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [term 1 LEADER]: Becoming Leader. State: Replica: b9e08d1f51e9423b8e73ddf2093811d6, State: Running, Role: LEADER
I20260810 06:49:02.873461 14385 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e08d1f51e9423b8e73ddf2093811d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } }
I20260810 06:49:02.874516 14367 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:02.876341 14387 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9e08d1f51e9423b8e73ddf2093811d6. Latest consensus state: current_term: 1 leader_uuid: "b9e08d1f51e9423b8e73ddf2093811d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e08d1f51e9423b8e73ddf2093811d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } } }
I20260810 06:49:02.876372 14386 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9e08d1f51e9423b8e73ddf2093811d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9e08d1f51e9423b8e73ddf2093811d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45305 } } }
I20260810 06:49:02.876758 14387 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:02.876766 14386 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:02.878579 14392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:02.883870 14392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:02.891887 14392 catalog_manager.cc:1347] Generated new cluster ID: b258984a1e134799aa6a67f9007fb635
I20260810 06:49:02.891981 14392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:02.899658 14392 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:02.900111 14392 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:02.906875 14392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9e08d1f51e9423b8e73ddf2093811d6: Generated new TSK 0
I20260810 06:49:02.907804 14392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:02.922554 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14262
W20260810 06:49:02.923492 14360 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.236:39727
W20260810 06:49:02.924124 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14262: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:02.924245 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 14262
2026-08-10T06:49:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2056 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:03 dist-test-slave-zj8q krb5kdc[14414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:03 dist-test-slave-zj8q krb5kdc[14414](info): set up 2 sockets
Aug 10 06:49:03 dist-test-slave-zj8q krb5kdc[14414](info): commencing operation
krb5kdc: starting...
W20260810 06:49:04.168282 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:04 dist-test-slave-zj8q krb5kdc[14414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:04.461261 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44385
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:04.792371 14435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:04.792644 14435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:04.792775 14435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:04.799911 14435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:04.800020 14435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:04.800094 14435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:04.800171 14435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:04.800238 14435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:04.809814 14435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44385
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:04.811008 14435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:04.812289 14435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:04.823591 14449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:04.823688 14444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:04.823690 14447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:04.824676 14435 server_base.cc:1034] running on GCE node
I20260810 06:49:04.825423 14435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:04.826835 14435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:04.828058 14435 hybrid_clock.cc:648] HybridClock initialized: now 1786344544828007 us; error 36 us; skew 500 ppm
I20260810 06:49:04.828395 14435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:04 dist-test-slave-zj8q krb5kdc[14414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344544, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:04.836174 14435 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:04.837723 14435 webserver.cc:460] Webserver started at http://127.0.0.1:43175/ using document root <none> and password file <none>
I20260810 06:49:04.838431 14435 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:04.838536 14435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:04.838796 14435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:04.841684 14435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "6da48e00eeca4f79893f8907ab696f5d"
format_stamp: "Formatted at 2026-08-10 06:49:04 on dist-test-slave-zj8q"
server_key: "ded9d7d944b93714b878eecebece907c"
server_key_iv: "322f2b438cbf0d33966a48c798fb9dbb"
server_key_version: "encryptionkey@0"
I20260810 06:49:04.842470 14435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "6da48e00eeca4f79893f8907ab696f5d"
format_stamp: "Formatted at 2026-08-10 06:49:04 on dist-test-slave-zj8q"
server_key: "ded9d7d944b93714b878eecebece907c"
server_key_iv: "322f2b438cbf0d33966a48c798fb9dbb"
server_key_version: "encryptionkey@0"
I20260810 06:49:04.847388 14435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:49:04.850623 14461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:04.851836 14435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:49:04.852005 14435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "6da48e00eeca4f79893f8907ab696f5d"
format_stamp: "Formatted at 2026-08-10 06:49:04 on dist-test-slave-zj8q"
server_key: "ded9d7d944b93714b878eecebece907c"
server_key_iv: "322f2b438cbf0d33966a48c798fb9dbb"
server_key_version: "encryptionkey@0"
I20260810 06:49:04.852182 14435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:04.871775 14435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:04.875710 14435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:04.875962 14435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:04.893071 14435 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44385
I20260810 06:49:04.893095 14549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44385 every 8 connection(s)
I20260810 06:49:04.894380 14435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:04.896950 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 14435
I20260810 06:49:04.897130 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:04.897666 11208 external_mini_cluster.cc:1428] Setting key f4f3fdf36e931d3e9252c4e494e4ba56
I20260810 06:49:04.898228 14550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:49:04 dist-test-slave-zj8q krb5kdc[14414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:49:04.912360 14550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d: Bootstrap starting.
I20260810 06:49:04.915512 14550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:04.916688 14550 log.cc:824] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:04.919498 14550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d: No bootstrap required, opened a new log
I20260810 06:49:04.920148 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:04.899386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:32991 (local address 127.0.0.1:44385)
0810 06:49:04.900085 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:04.900095 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:04.900143 (+ 48us) server_negotiation.cc:407] Connection header received
0810 06:49:04.901592 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:04.901613 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:04.901986 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:04.902340 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:04.904156 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:04.905562 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:04.907351 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:04.907987 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:04.908110 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:04.912916 (+ 4806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:04.912924 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:04.912937 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:04.912970 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:04.916007 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:04.916912 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:04.916918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:04.916924 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:04.917085 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:04.917552 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:04.917556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:04.917558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:04.918038 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:04.918350 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:04.918853 (+ 503us) server_negotiation.cc:299] Negotiation successful
0810 06:49:04.919157 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":312,"threads_started":1}
I20260810 06:49:04.923707 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:04.927338 14550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da48e00eeca4f79893f8907ab696f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } }
I20260810 06:49:04.927807 14550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:04.927942 14550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6da48e00eeca4f79893f8907ab696f5d, State: Initialized, Role: FOLLOWER
I20260810 06:49:04.928633 14550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da48e00eeca4f79893f8907ab696f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } }
I20260810 06:49:04.928922 14550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:04.929051 14550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:04.929230 14550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:04.931720 14550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da48e00eeca4f79893f8907ab696f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } }
I20260810 06:49:04.932164 14550 leader_election.cc:302] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6da48e00eeca4f79893f8907ab696f5d; no voters:
I20260810 06:49:04.932790 14550 leader_election.cc:288] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:04.932883 14563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:04.933985 14563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [term 1 LEADER]: Becoming Leader. State: Replica: 6da48e00eeca4f79893f8907ab696f5d, State: Running, Role: LEADER
I20260810 06:49:04.934558 14563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da48e00eeca4f79893f8907ab696f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } }
I20260810 06:49:04.935043 14550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:04.937551 14564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6da48e00eeca4f79893f8907ab696f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da48e00eeca4f79893f8907ab696f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } } }
I20260810 06:49:04.937937 14564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:04.942589 14565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6da48e00eeca4f79893f8907ab696f5d. Latest consensus state: current_term: 1 leader_uuid: "6da48e00eeca4f79893f8907ab696f5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da48e00eeca4f79893f8907ab696f5d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } } }
I20260810 06:49:04.943326 14565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:04.945264 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:04.927958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:41151 (local address 127.0.0.1:44385)
0810 06:49:04.928137 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:04.928141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:04.930594 (+ 2453us) server_negotiation.cc:407] Connection header received
0810 06:49:04.930761 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:04.930766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:04.930831 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:04.930911 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:04.932306 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:04.933298 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:04.934921 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:04.935392 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:04.935497 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:04.939017 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:04.939024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:04.939030 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:04.939069 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:04.942468 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:04.943211 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:04.943215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:04.943218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:04.943440 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:04.943899 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:04.943902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:04.943904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:04.944183 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:04.944465 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:04.944829 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 06:49:04.944995 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:49:04.946763 14569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:04.951263 14569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:04.960083 14569 catalog_manager.cc:1347] Generated new cluster ID: c3c27e37aa9040208b3ad5a0f2eee2c6
I20260810 06:49:04.960196 14569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:04.966642 14569 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:04.967913 14569 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:04.980291 14569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6da48e00eeca4f79893f8907ab696f5d: Generated new TSK 0
I20260810 06:49:04.981052 14569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:04.989511 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14435
I20260810 06:49:04.991143 14543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:05.080559 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 14435
W20260810 06:49:05.105455 14409 connection.cc:535] client connection to 127.0.0.1:44385 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44385): Connection reset by peer (error 104)
2026-08-10T06:49:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2157 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:05 dist-test-slave-zj8q krb5kdc[14592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:05 dist-test-slave-zj8q krb5kdc[14592](info): set up 2 sockets
Aug 10 06:49:05 dist-test-slave-zj8q krb5kdc[14592](info): commencing operation
krb5kdc: starting...
W20260810 06:49:06.325876 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:06 dist-test-slave-zj8q krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:06.600952 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:06.930541 14606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:06.930805 14606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:06.930949 14606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:06.937894 14606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:06.938007 14606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:06.938086 14606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:06.938194 14606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:06.938256 14606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:06.947837 14606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40647
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:06.949007 14606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:06.950315 14606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:06.961547 14626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:06.961697 14625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:06.961818 14630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:06.962766 14606 server_base.cc:1034] running on GCE node
I20260810 06:49:06.963531 14606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:06.964847 14606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:06.966053 14606 hybrid_clock.cc:648] HybridClock initialized: now 1786344546966008 us; error 33 us; skew 500 ppm
I20260810 06:49:06.966429 14606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:06 dist-test-slave-zj8q krb5kdc[14592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344546, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:06.969362 14606 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:06.970767 14606 webserver.cc:460] Webserver started at http://127.0.0.1:35569/ using document root <none> and password file <none>
I20260810 06:49:06.971407 14606 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:06.971524 14606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:06.971817 14606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:06.974732 14606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "2cb6329cd6bf490baa1de605478350a1"
format_stamp: "Formatted at 2026-08-10 06:49:06 on dist-test-slave-zj8q"
server_key: "168279e0d7426efea44256f340352ee7"
server_key_iv: "34980abd40eff57baafb63690373ad87"
server_key_version: "encryptionkey@0"
I20260810 06:49:06.975474 14606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "2cb6329cd6bf490baa1de605478350a1"
format_stamp: "Formatted at 2026-08-10 06:49:06 on dist-test-slave-zj8q"
server_key: "168279e0d7426efea44256f340352ee7"
server_key_iv: "34980abd40eff57baafb63690373ad87"
server_key_version: "encryptionkey@0"
I20260810 06:49:06.980343 14606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:49:06.983598 14638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:06.984804 14606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:49:06.984962 14606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "2cb6329cd6bf490baa1de605478350a1"
format_stamp: "Formatted at 2026-08-10 06:49:06 on dist-test-slave-zj8q"
server_key: "168279e0d7426efea44256f340352ee7"
server_key_iv: "34980abd40eff57baafb63690373ad87"
server_key_version: "encryptionkey@0"
I20260810 06:49:06.985141 14606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:07.000659 14606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:07.004338 14606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:07.004580 14606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:07.021976 14606 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40647
I20260810 06:49:07.022034 14721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40647 every 8 connection(s)
I20260810 06:49:07.023325 14606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:07.026963 14722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:07.027084 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 14606
I20260810 06:49:07.027248 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:07.027815 11208 external_mini_cluster.cc:1428] Setting key 3ca853cafd6844d48e687cd96a1f04cd
I20260810 06:49:07.040194 14722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1: Bootstrap starting.
I20260810 06:49:07.040524 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:07.029512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:53999 (local address 127.0.0.1:40647)
0810 06:49:07.030311 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:07.030322 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:07.030366 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:49:07.031950 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:07.031971 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:07.032355 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:07.032671 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:07.033237 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:07.034607 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:07.036344 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:07.037010 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:07.037138 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:07.038377 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:07.038385 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:07.038398 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:07.038900 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:07.038972 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:07.039002 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:07.039116 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:07.039396 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 06:49:07.039559 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":353,"threads_started":1}
I20260810 06:49:07.043340 14722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:07.044143 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:07.044379 14722 log.cc:824] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:07.047410 14722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1: No bootstrap required, opened a new log
I20260810 06:49:07.055647 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:07.048927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:57979 (local address 127.0.0.1:40647)
0810 06:49:07.049104 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:07.049110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:07.049596 (+ 486us) server_negotiation.cc:407] Connection header received
0810 06:49:07.049765 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:07.049770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:07.049844 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:07.049953 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:07.050431 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:07.051563 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:07.052902 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:07.053606 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:07.053768 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:07.054875 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:07.054881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:07.054886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:07.054910 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:07.054968 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:07.054987 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:07.055122 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:07.055269 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 06:49:07.055319 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:49:07.056841 14722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb6329cd6bf490baa1de605478350a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40647 } }
I20260810 06:49:07.057271 14722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:07.057351 14722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb6329cd6bf490baa1de605478350a1, State: Initialized, Role: FOLLOWER
I20260810 06:49:07.057834 14722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb6329cd6bf490baa1de605478350a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40647 } }
I20260810 06:49:07.058038 14722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:07.058132 14722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:07.058323 14722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:07.060335 14722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb6329cd6bf490baa1de605478350a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40647 } }
I20260810 06:49:07.060757 14722 leader_election.cc:302] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cb6329cd6bf490baa1de605478350a1; no voters:
I20260810 06:49:07.061364 14722 leader_election.cc:288] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:07.061412 14743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:07.062049 14743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [term 1 LEADER]: Becoming Leader. State: Replica: 2cb6329cd6bf490baa1de605478350a1, State: Running, Role: LEADER
I20260810 06:49:07.062745 14743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb6329cd6bf490baa1de605478350a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40647 } }
I20260810 06:49:07.063508 14722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:07.065920 14744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cb6329cd6bf490baa1de605478350a1. Latest consensus state: current_term: 1 leader_uuid: "2cb6329cd6bf490baa1de605478350a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb6329cd6bf490baa1de605478350a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40647 } } }
I20260810 06:49:07.065896 14745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cb6329cd6bf490baa1de605478350a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cb6329cd6bf490baa1de605478350a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40647 } } }
I20260810 06:49:07.066434 14744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:07.066515 14745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:07.073293 14751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:07.077978 14751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:07.086640 14751 catalog_manager.cc:1347] Generated new cluster ID: 4d3d0cd86e0147159fe6568deff8595d
I20260810 06:49:07.086745 14751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:07.108906 14751 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:07.110195 14751 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:07.120342 14751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cb6329cd6bf490baa1de605478350a1: Generated new TSK 0
I20260810 06:49:07.121119 14751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:07.129769 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14606
W20260810 06:49:07.131690 14716 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.236:53999
W20260810 06:49:07.132432 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14606: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:07.132592 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 14606
W20260810 06:49:07.160964 14590 connection.cc:535] client connection to 127.0.0.1:40647 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40647): Connection reset by peer (error 104)
2026-08-10T06:49:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2055 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:07 dist-test-slave-zj8q krb5kdc[14781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:07 dist-test-slave-zj8q krb5kdc[14781](info): set up 2 sockets
Aug 10 06:49:07 dist-test-slave-zj8q krb5kdc[14781](info): commencing operation
krb5kdc: starting...
W20260810 06:49:08.382350 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:08 dist-test-slave-zj8q krb5kdc[14781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:49:08.662494 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:08.987795 14801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:08.988070 14801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:08.988171 14801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:08.994993 14801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:08.995092 14801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:08.995146 14801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:08.995205 14801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:08.995255 14801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:09.004596 14801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35251
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:09.005780 14801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:09.007069 14801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:09.018062 14814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:09.018076 14813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:09.018514 14816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:09.018814 14801 server_base.cc:1034] running on GCE node
I20260810 06:49:09.019533 14801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:09.020802 14801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:09.021999 14801 hybrid_clock.cc:648] HybridClock initialized: now 1786344549021956 us; error 35 us; skew 500 ppm
I20260810 06:49:09.022349 14801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:09 dist-test-slave-zj8q krb5kdc[14781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344549, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:09.025334 14801 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:49:09.026721 14801 webserver.cc:460] Webserver started at http://127.0.0.1:42169/ using document root <none> and password file <none>
I20260810 06:49:09.027343 14801 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:09.027449 14801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:09.027695 14801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:09.030545 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "f12c1bd5815140ccbede99edce22aa2b"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-zj8q"
server_key: "47f24d5c7406ab4b636ed672b6d7ada5"
server_key_iv: "14094e029642641771f320dde4542eb6"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.031258 14801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "f12c1bd5815140ccbede99edce22aa2b"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-zj8q"
server_key: "47f24d5c7406ab4b636ed672b6d7ada5"
server_key_iv: "14094e029642641771f320dde4542eb6"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.036232 14801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:49:09.039538 14835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:09.040761 14801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:49:09.040917 14801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "f12c1bd5815140ccbede99edce22aa2b"
format_stamp: "Formatted at 2026-08-10 06:49:09 on dist-test-slave-zj8q"
server_key: "47f24d5c7406ab4b636ed672b6d7ada5"
server_key_iv: "14094e029642641771f320dde4542eb6"
server_key_version: "encryptionkey@0"
I20260810 06:49:09.041087 14801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:09.046466 14801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:09.050395 14801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:09.050657 14801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:09.066363 14801 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35251
I20260810 06:49:09.066402 14911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35251 every 8 connection(s)
I20260810 06:49:09.067617 14801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:09.068246 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 14801
I20260810 06:49:09.068442 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:09.068920 11208 external_mini_cluster.cc:1428] Setting key 6dd867765e2c81614944fc589cfd878f
I20260810 06:49:09.071935 14912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:09.077184 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.070576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:46693 (local address 127.0.0.1:35251)
0810 06:49:09.071318 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.071331 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.071383 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:49:09.073575 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.073600 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.073988 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.074417 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:09.075354 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:09.075366 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:09.075383 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:09.075401 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:09.075499 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:09.075530 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:09.075724 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.075850 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.076050 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":281,"threads_started":1}
I20260810 06:49:09.080564 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:09.084838 14912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b: Bootstrap starting.
I20260810 06:49:09.088039 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:09.085707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.236:54833 (local address 127.0.0.1:35251)
0810 06:49:09.085858 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:09.085863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:09.086351 (+ 488us) server_negotiation.cc:407] Connection header received
0810 06:49:09.086528 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:09.086532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:09.086577 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:09.086652 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:09.087476 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:09.087480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:09.087483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:09.087496 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:09.087530 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:09.087546 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:09.087625 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:09.087743 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 06:49:09.087787 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:49:09.088385 14912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:09.089376 14912 log.cc:824] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:09.091820 14912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b: No bootstrap required, opened a new log
I20260810 06:49:09.098544 14912 raft_consensus.cc:348] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12c1bd5815140ccbede99edce22aa2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35251 } }
I20260810 06:49:09.098894 14912 raft_consensus.cc:374] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:09.098989 14912 raft_consensus.cc:729] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f12c1bd5815140ccbede99edce22aa2b, State: Initialized, Role: FOLLOWER
I20260810 06:49:09.099469 14912 consensus_queue.cc:260] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12c1bd5815140ccbede99edce22aa2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35251 } }
I20260810 06:49:09.099687 14912 raft_consensus.cc:388] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:09.099833 14912 raft_consensus.cc:482] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:09.100025 14912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:09.101996 14912 raft_consensus.cc:504] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12c1bd5815140ccbede99edce22aa2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35251 } }
I20260810 06:49:09.102476 14912 leader_election.cc:302] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f12c1bd5815140ccbede99edce22aa2b; no voters:
I20260810 06:49:09.102982 14912 leader_election.cc:288] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:09.103091 14938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:09.103698 14938 raft_consensus.cc:686] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [term 1 LEADER]: Becoming Leader. State: Replica: f12c1bd5815140ccbede99edce22aa2b, State: Running, Role: LEADER
I20260810 06:49:09.104182 14938 consensus_queue.cc:237] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12c1bd5815140ccbede99edce22aa2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35251 } }
I20260810 06:49:09.104812 14912 sys_catalog.cc:564] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:09.107043 14940 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f12c1bd5815140ccbede99edce22aa2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12c1bd5815140ccbede99edce22aa2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35251 } } }
I20260810 06:49:09.106767 14942 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f12c1bd5815140ccbede99edce22aa2b. Latest consensus state: current_term: 1 leader_uuid: "f12c1bd5815140ccbede99edce22aa2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12c1bd5815140ccbede99edce22aa2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35251 } } }
I20260810 06:49:09.107471 14940 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:09.107471 14942 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:09.108409 14955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:09.113880 14955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:09.122362 14955 catalog_manager.cc:1347] Generated new cluster ID: 4331bf5b0888445f85e509867850aae1
I20260810 06:49:09.122474 14955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:09.138196 14955 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:09.138612 14955 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:09.147737 14955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f12c1bd5815140ccbede99edce22aa2b: Generated new TSK 0
I20260810 06:49:09.148504 14955 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:09.160593 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14801
W20260810 06:49:09.161685 14902 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.236:46693
W20260810 06:49:09.162353 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14801: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:09.162488 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 14801
2026-08-10T06:49:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2025 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:09 dist-test-slave-zj8q krb5kdc[14974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:09 dist-test-slave-zj8q krb5kdc[14974](info): set up 2 sockets
Aug 10 06:49:09 dist-test-slave-zj8q krb5kdc[14974](info): commencing operation
krb5kdc: starting...
W20260810 06:49:10.418445 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:10 dist-test-slave-zj8q krb5kdc[14974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:49:10.703054 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34495
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:40411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:34495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:11.030212 14997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:11.030486 14997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:11.030578 14997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:11.037540 14997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:11.037647 14997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:11.037703 14997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:11.037761 14997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:11.037804 14997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:11.047231 14997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:34495
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:11.048388 14997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:11.049726 14997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:11.060984 15009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:11.061067 15003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:11.061038 15004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:11.062194 14997 server_base.cc:1034] running on GCE node
I20260810 06:49:11.062928 14997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:11.064215 14997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:11.065413 14997 hybrid_clock.cc:648] HybridClock initialized: now 1786344551065366 us; error 41 us; skew 500 ppm
I20260810 06:49:11.065754 14997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:11 dist-test-slave-zj8q krb5kdc[14974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344551, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:11.068671 14997 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:49:11.069986 14997 webserver.cc:460] Webserver started at http://127.10.242.62:41493/ using document root <none> and password file <none>
I20260810 06:49:11.070672 14997 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:11.070793 14997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:11.071080 14997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:11.073921 14997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "96f17915300b4e3399fe682b1e298735"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-zj8q"
server_key: "dc3dca069d2ff86665e184c136134363"
server_key_iv: "5f14182bff93e41d3be47a23b624bf7a"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.074694 14997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "96f17915300b4e3399fe682b1e298735"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-zj8q"
server_key: "dc3dca069d2ff86665e184c136134363"
server_key_iv: "5f14182bff93e41d3be47a23b624bf7a"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.079459 14997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260810 06:49:11.082651 15020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:11.083801 14997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260810 06:49:11.083961 14997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "96f17915300b4e3399fe682b1e298735"
format_stamp: "Formatted at 2026-08-10 06:49:11 on dist-test-slave-zj8q"
server_key: "dc3dca069d2ff86665e184c136134363"
server_key_iv: "5f14182bff93e41d3be47a23b624bf7a"
server_key_version: "encryptionkey@0"
I20260810 06:49:11.084141 14997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:11.095449 14997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:11.098968 14997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:11.099223 14997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:11.116355 14997 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:34495
I20260810 06:49:11.116386 15116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:34495 every 8 connection(s)
I20260810 06:49:11.117673 14997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:11.118054 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 14997
I20260810 06:49:11.118248 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:11.118834 11208 external_mini_cluster.cc:1428] Setting key f617e02cb705d24c4fcbaeeb1c396949
I20260810 06:49:11.122210 15117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:49:11 dist-test-slave-zj8q krb5kdc[14974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:49:11.134508 15117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735: Bootstrap starting.
I20260810 06:49:11.138731 15117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:11.139909 15117 log.cc:824] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:11.142910 15117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735: No bootstrap required, opened a new log
I20260810 06:49:11.142937 15121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.120579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59879 (local address 127.10.242.62:34495)
0810 06:49:11.121322 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.121337 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.121389 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:49:11.123613 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.123642 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.124109 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.124502 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:11.125857 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.127885 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.129190 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.130213 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.130398 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.134726 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:11.134738 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:11.134762 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:11.134808 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:11.138595 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.139492 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.139502 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.139510 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.139744 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.140188 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.140192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.140194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.140760 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:11.141093 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.141601 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.141893 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":304,"threads_started":1}
I20260810 06:49:11.146767 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:11.151043 15117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f17915300b4e3399fe682b1e298735" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34495 } }
I20260810 06:49:11.151472 15117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:11.151567 15117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f17915300b4e3399fe682b1e298735, State: Initialized, Role: FOLLOWER
I20260810 06:49:11.152159 15117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f17915300b4e3399fe682b1e298735" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34495 } }
I20260810 06:49:11.152405 15117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:11.152545 15117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:11.152693 15117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:11.155352 15117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f17915300b4e3399fe682b1e298735" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34495 } }
I20260810 06:49:11.155805 15117 leader_election.cc:302] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96f17915300b4e3399fe682b1e298735; no voters:
I20260810 06:49:11.156464 15117 leader_election.cc:288] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:11.156569 15133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:11.157598 15133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [term 1 LEADER]: Becoming Leader. State: Replica: 96f17915300b4e3399fe682b1e298735, State: Running, Role: LEADER
I20260810 06:49:11.158182 15133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f17915300b4e3399fe682b1e298735" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34495 } }
I20260810 06:49:11.159910 15117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:11.160784 15133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96f17915300b4e3399fe682b1e298735. Latest consensus state: current_term: 1 leader_uuid: "96f17915300b4e3399fe682b1e298735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f17915300b4e3399fe682b1e298735" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34495 } } }
I20260810 06:49:11.160650 15137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96f17915300b4e3399fe682b1e298735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f17915300b4e3399fe682b1e298735" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34495 } } }
I20260810 06:49:11.161067 15137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:11.161078 15133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:11.168231 15121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:11.151131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45929 (local address 127.10.242.62:34495)
0810 06:49:11.151308 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:11.151314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:11.151726 (+ 412us) server_negotiation.cc:407] Connection header received
0810 06:49:11.151875 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:11.151885 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:11.151956 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:11.152055 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:11.153543 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.154674 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.155917 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:11.156654 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:11.156796 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:11.159770 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:11.159776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:11.159781 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:11.159816 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:11.163844 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.164815 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.164820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.164823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.165004 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:11.165599 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:11.165603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:11.165607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:11.166011 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:11.167479 (+ 1468us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:11.167658 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 06:49:11.167850 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:49:11.176103 15140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:11.180380 15140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:11.188472 15140 catalog_manager.cc:1347] Generated new cluster ID: 391c0a2160f9404189d7f362f90889f3
I20260810 06:49:11.188578 15140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:11.199110 15140 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:11.200834 15140 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:11.212280 15140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96f17915300b4e3399fe682b1e298735: Generated new TSK 0
I20260810 06:49:11.213281 15140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:11.220649 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 14997
I20260810 06:49:11.222172 15103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:11.311483 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 14997
W20260810 06:49:11.331763 14970 connection.cc:535] client connection to 127.10.242.62:34495 recv error: Network error: failed to read from TLS socket (remote: 127.10.242.62:34495): Connection reset by peer (error 104)
2026-08-10T06:49:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2144 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:11 dist-test-slave-zj8q krb5kdc[15177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:11 dist-test-slave-zj8q krb5kdc[15177](info): set up 2 sockets
Aug 10 06:49:11 dist-test-slave-zj8q krb5kdc[15177](info): commencing operation
krb5kdc: starting...
W20260810 06:49:12.543521 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:12 dist-test-slave-zj8q krb5kdc[15177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:49:12.831486 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:38867
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:38363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:38867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:13.177078 15198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:13.177347 15198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:13.177480 15198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:13.184557 15198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:13.184691 15198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:13.184752 15198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:13.184834 15198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:13.184890 15198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:13.194620 15198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:38363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:38867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:38867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:13.195883 15198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:13.197216 15198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:13.208343 15204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:13.208441 15211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:13.208413 15206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:13.209414 15198 server_base.cc:1034] running on GCE node
I20260810 06:49:13.210135 15198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:13.211504 15198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:13.212714 15198 hybrid_clock.cc:648] HybridClock initialized: now 1786344553212661 us; error 37 us; skew 500 ppm
I20260810 06:49:13.213029 15198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:13 dist-test-slave-zj8q krb5kdc[15177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344553, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:13.215976 15198 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:49:13.217306 15198 webserver.cc:460] Webserver started at http://127.10.242.62:44289/ using document root <none> and password file <none>
I20260810 06:49:13.217945 15198 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:13.218063 15198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:13.218380 15198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:13.221302 15198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "7856bac3b44a4516a082065f3321635b"
format_stamp: "Formatted at 2026-08-10 06:49:13 on dist-test-slave-zj8q"
server_key: "b7e022ee9e4ffa763eda4553eb9cdb66"
server_key_iv: "e091d8f67ff11b1234e51b22cb9d7b62"
server_key_version: "encryptionkey@0"
I20260810 06:49:13.222016 15198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "7856bac3b44a4516a082065f3321635b"
format_stamp: "Formatted at 2026-08-10 06:49:13 on dist-test-slave-zj8q"
server_key: "b7e022ee9e4ffa763eda4553eb9cdb66"
server_key_iv: "e091d8f67ff11b1234e51b22cb9d7b62"
server_key_version: "encryptionkey@0"
I20260810 06:49:13.226850 15198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260810 06:49:13.230113 15222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:13.231379 15198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:49:13.231528 15198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "7856bac3b44a4516a082065f3321635b"
format_stamp: "Formatted at 2026-08-10 06:49:13 on dist-test-slave-zj8q"
server_key: "b7e022ee9e4ffa763eda4553eb9cdb66"
server_key_iv: "e091d8f67ff11b1234e51b22cb9d7b62"
server_key_version: "encryptionkey@0"
I20260810 06:49:13.231699 15198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:13.249014 15198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:13.252975 15198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:13.253232 15198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:13.270766 15198 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:38867
I20260810 06:49:13.270805 15317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:38867 every 8 connection(s)
I20260810 06:49:13.272097 15198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:13.275858 15319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:13.280011 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 15198
I20260810 06:49:13.280237 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:13.280790 11208 external_mini_cluster.cc:1428] Setting key 9dca08c4b465d05c14f06f79c1b6f14c
I20260810 06:49:13.288921 15319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b: Bootstrap starting.
I20260810 06:49:13.292457 15319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:13.293613 15319 log.cc:824] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:13.293741 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:13.282617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53041 (local address 127.10.242.62:38867)
0810 06:49:13.283371 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:13.283382 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:13.283425 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:49:13.285016 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:13.285039 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:13.285473 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:13.285817 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:13.286448 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:13.287781 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:13.289541 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:13.290229 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:13.290358 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:13.291588 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:13.291595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:13.291608 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:13.292100 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:13.292173 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:13.292208 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:13.292321 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:13.292632 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 06:49:13.292789 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":313,"threads_started":1}
I20260810 06:49:13.296407 15319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b: No bootstrap required, opened a new log
I20260810 06:49:13.297272 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:13.304492 15319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7856bac3b44a4516a082065f3321635b" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38867 } }
I20260810 06:49:13.304919 15319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:13.305027 15319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7856bac3b44a4516a082065f3321635b, State: Initialized, Role: FOLLOWER
I20260810 06:49:13.305694 15319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7856bac3b44a4516a082065f3321635b" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38867 } }
I20260810 06:49:13.305981 15319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:13.306131 15319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:13.306337 15319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:13.307896 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:13.301470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43837 (local address 127.10.242.62:38867)
0810 06:49:13.301641 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:13.301646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:13.301936 (+ 290us) server_negotiation.cc:407] Connection header received
0810 06:49:13.302089 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:13.302092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:13.302143 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:13.302264 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:13.302811 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:13.303552 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:13.305389 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:13.306034 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:13.306209 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:13.307207 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:13.307213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:13.307218 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:13.307241 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:13.307298 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:13.307323 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:13.307436 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:13.307539 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:49:13.307586 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:49:13.309134 15319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7856bac3b44a4516a082065f3321635b" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38867 } }
I20260810 06:49:13.309594 15319 leader_election.cc:302] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7856bac3b44a4516a082065f3321635b; no voters:
I20260810 06:49:13.310096 15319 leader_election.cc:288] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:13.310256 15333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:13.311345 15333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [term 1 LEADER]: Becoming Leader. State: Replica: 7856bac3b44a4516a082065f3321635b, State: Running, Role: LEADER
I20260810 06:49:13.311878 15333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7856bac3b44a4516a082065f3321635b" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38867 } }
I20260810 06:49:13.312270 15319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:13.314613 15337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7856bac3b44a4516a082065f3321635b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7856bac3b44a4516a082065f3321635b" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38867 } } }
I20260810 06:49:13.315033 15337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:13.316205 15339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7856bac3b44a4516a082065f3321635b. Latest consensus state: current_term: 1 leader_uuid: "7856bac3b44a4516a082065f3321635b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7856bac3b44a4516a082065f3321635b" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 38867 } } }
I20260810 06:49:13.316540 15339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:13.317457 15348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:13.321530 15348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:13.329485 15348 catalog_manager.cc:1347] Generated new cluster ID: 19f9e61115834d65b3a5124320c8a77e
I20260810 06:49:13.329591 15348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:13.339803 15348 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:13.341564 15348 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:13.356734 15348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7856bac3b44a4516a082065f3321635b: Generated new TSK 0
I20260810 06:49:13.357481 15348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:13.373807 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15198
W20260810 06:49:13.374997 15311 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53041
W20260810 06:49:13.375774 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15198: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:13.375913 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 15198
W20260810 06:49:13.401885 15175 connection.cc:535] client connection to 127.10.242.62:38867 recv error: Network error: failed to read from TLS socket (remote: 127.10.242.62:38867): Connection reset by peer (error 104)
2026-08-10T06:49:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2070 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:13 dist-test-slave-zj8q krb5kdc[15364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:13 dist-test-slave-zj8q krb5kdc[15364](info): set up 2 sockets
Aug 10 06:49:13 dist-test-slave-zj8q krb5kdc[15364](info): commencing operation
krb5kdc: starting...
W20260810 06:49:14.622550 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:14 dist-test-slave-zj8q krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:49:14.913393 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:37051
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:42783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:37051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:15.241976 15384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:15.242270 15384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:15.242401 15384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:15.249723 15384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:15.249825 15384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:15.249903 15384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:15.249986 15384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:15.250053 15384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:15.260483 15384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:42783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:37051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:37051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:15.261749 15384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:15.263073 15384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:15.274274 15392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:15.274312 15394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:15.274237 15400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:15.275256 15384 server_base.cc:1034] running on GCE node
I20260810 06:49:15.275992 15384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:15.277340 15384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:15.278538 15384 hybrid_clock.cc:648] HybridClock initialized: now 1786344555278479 us; error 45 us; skew 500 ppm
I20260810 06:49:15.278872 15384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:15 dist-test-slave-zj8q krb5kdc[15364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344555, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:15.285933 15384 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:49:15.287402 15384 webserver.cc:460] Webserver started at http://127.10.242.62:42265/ using document root <none> and password file <none>
I20260810 06:49:15.288040 15384 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:15.288160 15384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:15.288452 15384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:15.291388 15384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "eb8d26e3a59b4b8eb5db63206f7110cf"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-zj8q"
server_key: "36c5bddc0d0e00c0125a7968cdcc855d"
server_key_iv: "66d54e0970efa720203ec442141844d3"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.292114 15384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "eb8d26e3a59b4b8eb5db63206f7110cf"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-zj8q"
server_key: "36c5bddc0d0e00c0125a7968cdcc855d"
server_key_iv: "66d54e0970efa720203ec442141844d3"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.297101 15384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:49:15.300390 15409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:15.301664 15384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:49:15.301826 15384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "eb8d26e3a59b4b8eb5db63206f7110cf"
format_stamp: "Formatted at 2026-08-10 06:49:15 on dist-test-slave-zj8q"
server_key: "36c5bddc0d0e00c0125a7968cdcc855d"
server_key_iv: "66d54e0970efa720203ec442141844d3"
server_key_version: "encryptionkey@0"
I20260810 06:49:15.302002 15384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:15.307612 15384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:15.311369 15384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:15.311636 15384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:15.328908 15384 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:37051
I20260810 06:49:15.328938 15500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:37051 every 8 connection(s)
I20260810 06:49:15.330403 15384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:15.333959 15502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:15.337697 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 15384
I20260810 06:49:15.337862 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:15.338325 11208 external_mini_cluster.cc:1428] Setting key 1cef97f627242aea38705342e7e6af77
I20260810 06:49:15.346321 15506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.340035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42691 (local address 127.10.242.62:37051)
0810 06:49:15.340812 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.340826 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.340877 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:49:15.342604 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.342633 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.343070 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.343488 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:15.344436 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:15.344444 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:15.344464 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:15.344486 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:15.344548 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:15.344578 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:15.344802 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.344940 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.345114 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":306,"threads_started":1}
I20260810 06:49:15.347038 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf: Bootstrap starting.
I20260810 06:49:15.349658 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:15.350049 15502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:15.351014 15502 log.cc:824] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:15.353798 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf: No bootstrap required, opened a new log
I20260810 06:49:15.356175 15506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:15.353802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57885 (local address 127.10.242.62:37051)
0810 06:49:15.353948 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:15.353954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:15.354341 (+ 387us) server_negotiation.cc:407] Connection header received
0810 06:49:15.354503 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:15.354506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:15.354552 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:15.354627 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:15.355538 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:15.355544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:15.355547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:15.355562 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:15.355601 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:15.355615 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:15.355724 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:15.355837 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 06:49:15.355893 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:49:15.361567 15502 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8d26e3a59b4b8eb5db63206f7110cf" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37051 } }
I20260810 06:49:15.361943 15502 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:15.362030 15502 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb8d26e3a59b4b8eb5db63206f7110cf, State: Initialized, Role: FOLLOWER
I20260810 06:49:15.362574 15502 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8d26e3a59b4b8eb5db63206f7110cf" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37051 } }
I20260810 06:49:15.362798 15502 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:15.362923 15502 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:15.363071 15502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:15.365089 15502 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8d26e3a59b4b8eb5db63206f7110cf" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37051 } }
I20260810 06:49:15.365599 15502 leader_election.cc:302] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb8d26e3a59b4b8eb5db63206f7110cf; no voters:
I20260810 06:49:15.366125 15502 leader_election.cc:288] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:15.366271 15525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:15.366838 15525 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [term 1 LEADER]: Becoming Leader. State: Replica: eb8d26e3a59b4b8eb5db63206f7110cf, State: Running, Role: LEADER
I20260810 06:49:15.367368 15525 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8d26e3a59b4b8eb5db63206f7110cf" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37051 } }
I20260810 06:49:15.368054 15502 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:15.370119 15528 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb8d26e3a59b4b8eb5db63206f7110cf. Latest consensus state: current_term: 1 leader_uuid: "eb8d26e3a59b4b8eb5db63206f7110cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8d26e3a59b4b8eb5db63206f7110cf" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37051 } } }
I20260810 06:49:15.370561 15528 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:15.371032 15527 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb8d26e3a59b4b8eb5db63206f7110cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb8d26e3a59b4b8eb5db63206f7110cf" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37051 } } }
I20260810 06:49:15.371383 15527 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:15.374594 15536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:15.379000 15536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:15.386667 15536 catalog_manager.cc:1347] Generated new cluster ID: c94fb6cf6a604beda097697a76d1d607
I20260810 06:49:15.386768 15536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:15.398672 15536 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:15.399178 15536 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:15.409667 15536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb8d26e3a59b4b8eb5db63206f7110cf: Generated new TSK 0
I20260810 06:49:15.410408 15536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:15.420055 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15384
W20260810 06:49:15.421046 15496 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42691
W20260810 06:49:15.421679 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15384: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:15.421806 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 15384
2026-08-10T06:49:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2044 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:15 dist-test-slave-zj8q krb5kdc[15563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:15 dist-test-slave-zj8q krb5kdc[15563](info): set up 2 sockets
Aug 10 06:49:15 dist-test-slave-zj8q krb5kdc[15563](info): commencing operation
krb5kdc: starting...
W20260810 06:49:16.659159 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.007s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:16 dist-test-slave-zj8q krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:49:16.952718 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:37805
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:35839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:37805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:17.280967 15584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:17.281227 15584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:17.281325 15584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:17.288269 15584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:17.288372 15584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:17.288425 15584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:17.288487 15584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:17.288532 15584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:17.297828 15584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:35839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:37805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:37805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:17.298980 15584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:17.300251 15584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:17.311762 15596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:17.311728 15603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:17.311707 15600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:17.312587 15584 server_base.cc:1034] running on GCE node
I20260810 06:49:17.313328 15584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:17.314739 15584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:17.315958 15584 hybrid_clock.cc:648] HybridClock initialized: now 1786344557315897 us; error 41 us; skew 500 ppm
I20260810 06:49:17.316318 15584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:17 dist-test-slave-zj8q krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344557, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:17.319283 15584 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:49:17.320670 15584 webserver.cc:460] Webserver started at http://127.10.242.62:38627/ using document root <none> and password file <none>
I20260810 06:49:17.321331 15584 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:17.321451 15584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:17.321749 15584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:17.324784 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "177b78afd3c84e60957cc71d95fa520a"
format_stamp: "Formatted at 2026-08-10 06:49:17 on dist-test-slave-zj8q"
server_key: "3cb660d57e131992cc33ab379f586eb8"
server_key_iv: "251a6a26b1df9b28cec3decd0643ede0"
server_key_version: "encryptionkey@0"
I20260810 06:49:17.325574 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "177b78afd3c84e60957cc71d95fa520a"
format_stamp: "Formatted at 2026-08-10 06:49:17 on dist-test-slave-zj8q"
server_key: "3cb660d57e131992cc33ab379f586eb8"
server_key_iv: "251a6a26b1df9b28cec3decd0643ede0"
server_key_version: "encryptionkey@0"
I20260810 06:49:17.330873 15584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 06:49:17.334405 15612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:17.335652 15584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:49:17.335819 15584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "177b78afd3c84e60957cc71d95fa520a"
format_stamp: "Formatted at 2026-08-10 06:49:17 on dist-test-slave-zj8q"
server_key: "3cb660d57e131992cc33ab379f586eb8"
server_key_iv: "251a6a26b1df9b28cec3decd0643ede0"
server_key_version: "encryptionkey@0"
I20260810 06:49:17.335987 15584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:17.353109 15584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:17.356861 15584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:17.357120 15584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:17.374142 15584 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:37805
I20260810 06:49:17.374172 15712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:37805 every 8 connection(s)
I20260810 06:49:17.375442 15584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:17.377259 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 15584
I20260810 06:49:17.377445 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:17.378011 11208 external_mini_cluster.cc:1428] Setting key 169c4aff543933b8e619811db5724492
I20260810 06:49:17.379768 15714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:49:17 dist-test-slave-zj8q krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:49:17.392565 15714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a: Bootstrap starting.
I20260810 06:49:17.395965 15714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:17.397063 15714 log.cc:824] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:17.400238 15714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a: No bootstrap required, opened a new log
I20260810 06:49:17.400964 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:17.379807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35217 (local address 127.10.242.62:37805)
0810 06:49:17.380591 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:17.380603 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:17.380646 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:49:17.382327 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:17.382349 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:17.382754 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:17.383086 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:17.384849 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:17.386353 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:17.388029 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:17.388720 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:17.388863 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:17.393595 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:17.393602 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:17.393615 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:17.393655 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:17.396646 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:17.397469 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:17.397476 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:17.397482 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:17.397806 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:17.398224 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:17.398229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:17.398233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:17.398816 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:17.399028 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:17.399572 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 06:49:17.399900 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":353,"threads_started":1}
I20260810 06:49:17.404639 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:17.409688 15714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "177b78afd3c84e60957cc71d95fa520a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37805 } }
I20260810 06:49:17.410022 15714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:17.410125 15714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177b78afd3c84e60957cc71d95fa520a, State: Initialized, Role: FOLLOWER
I20260810 06:49:17.410714 15714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "177b78afd3c84e60957cc71d95fa520a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37805 } }
I20260810 06:49:17.410923 15714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:17.411048 15714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:17.411204 15714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:17.413272 15714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "177b78afd3c84e60957cc71d95fa520a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37805 } }
I20260810 06:49:17.413724 15714 leader_election.cc:302] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 177b78afd3c84e60957cc71d95fa520a; no voters:
I20260810 06:49:17.414324 15714 leader_election.cc:288] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:17.414426 15735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:17.415737 15735 raft_consensus.cc:686] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [term 1 LEADER]: Becoming Leader. State: Replica: 177b78afd3c84e60957cc71d95fa520a, State: Running, Role: LEADER
I20260810 06:49:17.416257 15735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "177b78afd3c84e60957cc71d95fa520a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37805 } }
I20260810 06:49:17.416482 15714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:17.419662 15739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "177b78afd3c84e60957cc71d95fa520a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "177b78afd3c84e60957cc71d95fa520a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37805 } } }
I20260810 06:49:17.420063 15739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:17.424510 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:17.409507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36129 (local address 127.10.242.62:37805)
0810 06:49:17.409673 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:17.409679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:17.410212 (+ 533us) server_negotiation.cc:407] Connection header received
0810 06:49:17.410374 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:17.410380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:17.410438 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:17.410513 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:17.412025 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:17.412753 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:17.414018 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:17.414636 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:17.414774 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:17.417297 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:17.417302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:17.417305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:17.417334 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:17.421011 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:17.421889 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:17.421896 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:17.421900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:17.422090 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:17.422587 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:17.422592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:17.422596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:17.422978 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:17.423931 (+ 953us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:17.424099 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:49:17.424266 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:49:17.424757 15740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 177b78afd3c84e60957cc71d95fa520a. Latest consensus state: current_term: 1 leader_uuid: "177b78afd3c84e60957cc71d95fa520a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "177b78afd3c84e60957cc71d95fa520a" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 37805 } } }
I20260810 06:49:17.425060 15740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:17.426748 15746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:17.432569 15746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:17.441871 15746 catalog_manager.cc:1347] Generated new cluster ID: 603705fd272a4bc8adc2700d72f8dfaf
I20260810 06:49:17.441977 15746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:17.453068 15746 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:17.454250 15746 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:17.459496 15746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 177b78afd3c84e60957cc71d95fa520a: Generated new TSK 0
I20260810 06:49:17.460139 15746 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:17.469877 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15584
I20260810 06:49:17.470996 15706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:17.557585 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 15584
2026-08-10T06:49:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2133 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:17 dist-test-slave-zj8q krb5kdc[15781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:17 dist-test-slave-zj8q krb5kdc[15781](info): set up 2 sockets
Aug 10 06:49:17 dist-test-slave-zj8q krb5kdc[15781](info): commencing operation
krb5kdc: starting...
W20260810 06:49:18.807519 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:18 dist-test-slave-zj8q krb5kdc[15781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:49:19.091116 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34623
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:41799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:34623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:19.423302 15808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:19.423595 15808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:19.423738 15808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:19.430699 15808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:19.430812 15808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:19.430891 15808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:19.430979 15808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:19.431039 15808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:19.440433 15808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:41799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:34623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:34623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:19.441684 15808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:19.442994 15808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:19.454030 15820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:19.454094 15816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:19.454053 15825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:19.454772 15808 server_base.cc:1034] running on GCE node
I20260810 06:49:19.455526 15808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:19.456841 15808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:19.458057 15808 hybrid_clock.cc:648] HybridClock initialized: now 1786344559458011 us; error 28 us; skew 500 ppm
I20260810 06:49:19.458395 15808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:19 dist-test-slave-zj8q krb5kdc[15781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344559, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:19.461369 15808 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:49:19.462813 15808 webserver.cc:460] Webserver started at http://127.10.242.62:43027/ using document root <none> and password file <none>
I20260810 06:49:19.463425 15808 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:19.463553 15808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:19.463850 15808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:19.466740 15808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "2831fa2007144d0fba9c0234a1a55d63"
format_stamp: "Formatted at 2026-08-10 06:49:19 on dist-test-slave-zj8q"
server_key: "9e5695e7d21850ed337aa3d2fb76dee2"
server_key_iv: "346c188423af2ef78f4e14a563c5fd5c"
server_key_version: "encryptionkey@0"
I20260810 06:49:19.467458 15808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "2831fa2007144d0fba9c0234a1a55d63"
format_stamp: "Formatted at 2026-08-10 06:49:19 on dist-test-slave-zj8q"
server_key: "9e5695e7d21850ed337aa3d2fb76dee2"
server_key_iv: "346c188423af2ef78f4e14a563c5fd5c"
server_key_version: "encryptionkey@0"
I20260810 06:49:19.472290 15808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:49:19.475674 15841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:19.476888 15808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:49:19.477044 15808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "2831fa2007144d0fba9c0234a1a55d63"
format_stamp: "Formatted at 2026-08-10 06:49:19 on dist-test-slave-zj8q"
server_key: "9e5695e7d21850ed337aa3d2fb76dee2"
server_key_iv: "346c188423af2ef78f4e14a563c5fd5c"
server_key_version: "encryptionkey@0"
I20260810 06:49:19.477213 15808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:19.506733 15808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:19.510780 15808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:19.511052 15808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:19.528324 15808 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:34623
I20260810 06:49:19.528347 15935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:34623 every 8 connection(s)
I20260810 06:49:19.529667 15808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:19.533097 15938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:19.536938 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 15808
I20260810 06:49:19.537111 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:19.537635 11208 external_mini_cluster.cc:1428] Setting key b47cbfcdf8327ac7195089f8d15cf4c8
I20260810 06:49:19.545593 15938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63: Bootstrap starting.
I20260810 06:49:19.548591 15938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:19.549659 15938 log.cc:824] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:19.550637 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:19.539849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37773 (local address 127.10.242.62:34623)
0810 06:49:19.540720 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:19.540731 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:19.540772 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:49:19.542387 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:19.542410 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:19.542820 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:19.543167 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:19.543723 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:19.544999 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:19.546406 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:19.547103 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:19.547295 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:19.548359 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:19.548366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:19.548380 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:19.548952 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:19.549030 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:19.549056 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:19.549154 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:19.549378 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 06:49:19.549621 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":375,"threads_started":1}
I20260810 06:49:19.552273 15938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63: No bootstrap required, opened a new log
I20260810 06:49:19.553946 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:19.561721 15938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2831fa2007144d0fba9c0234a1a55d63" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34623 } }
I20260810 06:49:19.562060 15938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:19.562168 15938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2831fa2007144d0fba9c0234a1a55d63, State: Initialized, Role: FOLLOWER
I20260810 06:49:19.562691 15938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2831fa2007144d0fba9c0234a1a55d63" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34623 } }
I20260810 06:49:19.562911 15938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:19.563047 15938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:19.563225 15938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:19.565685 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:19.559046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52101 (local address 127.10.242.62:34623)
0810 06:49:19.559219 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:19.559224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:19.559725 (+ 501us) server_negotiation.cc:407] Connection header received
0810 06:49:19.559859 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:19.559864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:19.559941 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:19.560057 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:19.560446 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:19.561482 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:19.563020 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:19.563672 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:19.563799 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:19.564771 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:19.564777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:19.564782 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:19.564803 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:19.564847 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:19.564865 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:19.564958 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:19.565325 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 06:49:19.565411 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:49:19.565671 15938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2831fa2007144d0fba9c0234a1a55d63" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34623 } }
I20260810 06:49:19.566320 15938 leader_election.cc:302] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2831fa2007144d0fba9c0234a1a55d63; no voters:
I20260810 06:49:19.566876 15938 leader_election.cc:288] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:19.567059 15959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:19.567744 15959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [term 1 LEADER]: Becoming Leader. State: Replica: 2831fa2007144d0fba9c0234a1a55d63, State: Running, Role: LEADER
I20260810 06:49:19.568254 15959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2831fa2007144d0fba9c0234a1a55d63" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34623 } }
I20260810 06:49:19.568830 15938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:19.570971 15961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2831fa2007144d0fba9c0234a1a55d63. Latest consensus state: current_term: 1 leader_uuid: "2831fa2007144d0fba9c0234a1a55d63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2831fa2007144d0fba9c0234a1a55d63" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34623 } } }
I20260810 06:49:19.571388 15961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:19.572269 15960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2831fa2007144d0fba9c0234a1a55d63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2831fa2007144d0fba9c0234a1a55d63" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 34623 } } }
I20260810 06:49:19.572933 15960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:19.578212 15977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:19.582777 15977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:19.591001 15977 catalog_manager.cc:1347] Generated new cluster ID: 211dbe290c5d48e69f0a5f7164160197
I20260810 06:49:19.591105 15977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:19.617240 15977 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:19.618618 15977 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:19.631771 15977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2831fa2007144d0fba9c0234a1a55d63: Generated new TSK 0
I20260810 06:49:19.632537 15977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:19.638432 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15808
W20260810 06:49:19.639513 15928 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37773
W20260810 06:49:19.640113 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 15808: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:19.640254 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 15808
2026-08-10T06:49:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2085 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:19 dist-test-slave-zj8q krb5kdc[16006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:19 dist-test-slave-zj8q krb5kdc[16006](info): set up 2 sockets
Aug 10 06:49:19 dist-test-slave-zj8q krb5kdc[16006](info): commencing operation
krb5kdc: starting...
W20260810 06:49:20.888980 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:21 dist-test-slave-zj8q krb5kdc[16006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:49:21.178473 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:39381
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:46811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:39381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:21.512113 16029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:21.512387 16029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:21.512516 16029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:21.519629 16029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:21.519732 16029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:21.519788 16029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:21.519873 16029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:21.519932 16029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:21.529484 16029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:46811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:39381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:39381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:21.530781 16029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:21.532052 16029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:21.543421 16039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:21.543495 16040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:21.543654 16042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:21.544370 16029 server_base.cc:1034] running on GCE node
I20260810 06:49:21.545127 16029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:21.546496 16029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:21.547727 16029 hybrid_clock.cc:648] HybridClock initialized: now 1786344561547662 us; error 43 us; skew 500 ppm
I20260810 06:49:21.548050 16029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:21 dist-test-slave-zj8q krb5kdc[16006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344561, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:21.553704 16029 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:49:21.555137 16029 webserver.cc:460] Webserver started at http://127.10.242.62:34239/ using document root <none> and password file <none>
I20260810 06:49:21.555794 16029 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:21.555914 16029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:21.556200 16029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:21.559139 16029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "998466b731f04713bc1d7f1744a9bc60"
format_stamp: "Formatted at 2026-08-10 06:49:21 on dist-test-slave-zj8q"
server_key: "355761f2712013b49f4c562a7d75b117"
server_key_iv: "30e1090c186449ab5a80839eb1f38ec6"
server_key_version: "encryptionkey@0"
I20260810 06:49:21.559872 16029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "998466b731f04713bc1d7f1744a9bc60"
format_stamp: "Formatted at 2026-08-10 06:49:21 on dist-test-slave-zj8q"
server_key: "355761f2712013b49f4c562a7d75b117"
server_key_iv: "30e1090c186449ab5a80839eb1f38ec6"
server_key_version: "encryptionkey@0"
I20260810 06:49:21.564958 16029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:49:21.568364 16050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:21.569880 16029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:49:21.570026 16029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "998466b731f04713bc1d7f1744a9bc60"
format_stamp: "Formatted at 2026-08-10 06:49:21 on dist-test-slave-zj8q"
server_key: "355761f2712013b49f4c562a7d75b117"
server_key_iv: "30e1090c186449ab5a80839eb1f38ec6"
server_key_version: "encryptionkey@0"
I20260810 06:49:21.570276 16029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:21.575608 16029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:21.579279 16029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:21.579519 16029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:21.595875 16029 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:39381
I20260810 06:49:21.595911 16134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:39381 every 8 connection(s)
I20260810 06:49:21.597172 16029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:21.600755 16138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:21.605222 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16029
I20260810 06:49:21.605382 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:21.605816 11208 external_mini_cluster.cc:1428] Setting key 1f7d4bd85b0a399eb5667c00575f9b3d
I20260810 06:49:21.611163 16138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60: Bootstrap starting.
I20260810 06:49:21.613085 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:21.607395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40341 (local address 127.10.242.62:39381)
0810 06:49:21.608165 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:21.608186 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:21.608231 (+ 45us) server_negotiation.cc:407] Connection header received
0810 06:49:21.609951 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:21.609972 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:21.610319 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:21.610610 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:21.611645 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:21.611655 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:21.611667 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:21.611681 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:21.611740 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:21.611759 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:21.611916 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:21.612023 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 06:49:21.612201 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":339,"threads_started":1}
I20260810 06:49:21.614686 16138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:21.615943 16138 log.cc:824] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:21.616935 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:21.618590 16138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60: No bootstrap required, opened a new log
I20260810 06:49:21.623418 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:21.621008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45273 (local address 127.10.242.62:39381)
0810 06:49:21.621197 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:21.621202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:21.621543 (+ 341us) server_negotiation.cc:407] Connection header received
0810 06:49:21.621696 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:21.621700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:21.621748 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:21.621826 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:21.622785 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:21.622788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:21.622791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:49:21.622805 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:21.622845 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:49:21.622860 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:21.622951 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:21.623071 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 06:49:21.623119 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:49:21.627000 16138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998466b731f04713bc1d7f1744a9bc60" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39381 } }
I20260810 06:49:21.627327 16138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:21.627417 16138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998466b731f04713bc1d7f1744a9bc60, State: Initialized, Role: FOLLOWER
I20260810 06:49:21.627951 16138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998466b731f04713bc1d7f1744a9bc60" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39381 } }
I20260810 06:49:21.628180 16138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:21.628309 16138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:21.628480 16138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:21.630676 16138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998466b731f04713bc1d7f1744a9bc60" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39381 } }
I20260810 06:49:21.631138 16138 leader_election.cc:302] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 998466b731f04713bc1d7f1744a9bc60; no voters:
I20260810 06:49:21.631660 16138 leader_election.cc:288] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:21.631778 16158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:21.632375 16158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [term 1 LEADER]: Becoming Leader. State: Replica: 998466b731f04713bc1d7f1744a9bc60, State: Running, Role: LEADER
I20260810 06:49:21.632853 16158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998466b731f04713bc1d7f1744a9bc60" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39381 } }
I20260810 06:49:21.633541 16138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:21.635825 16160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998466b731f04713bc1d7f1744a9bc60. Latest consensus state: current_term: 1 leader_uuid: "998466b731f04713bc1d7f1744a9bc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998466b731f04713bc1d7f1744a9bc60" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39381 } } }
I20260810 06:49:21.635826 16159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "998466b731f04713bc1d7f1744a9bc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998466b731f04713bc1d7f1744a9bc60" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 39381 } } }
I20260810 06:49:21.636261 16160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:21.636257 16159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:21.637763 16169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:21.642014 16169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:21.649561 16169 catalog_manager.cc:1347] Generated new cluster ID: d7ed310a96ac4a07aa3a0db40b53c6b4
I20260810 06:49:21.649667 16169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:21.666460 16169 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:21.666836 16169 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:21.674196 16169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 998466b731f04713bc1d7f1744a9bc60: Generated new TSK 0
I20260810 06:49:21.674963 16169 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:21.687788 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16029
W20260810 06:49:21.689060 16128 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40341
W20260810 06:49:21.689710 11208 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16029: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:21.689832 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16029
2026-08-10T06:49:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2049 ms)
[----------] 18 tests from AuthTokenIssuingTest (37603 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:21 dist-test-slave-zj8q krb5kdc[16188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:21 dist-test-slave-zj8q krb5kdc[16188](info): set up 2 sockets
Aug 10 06:49:21 dist-test-slave-zj8q krb5kdc[16188](info): commencing operation
krb5kdc: starting...
W20260810 06:49:22.951647 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.003s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:23 dist-test-slave-zj8q krb5kdc[16188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:49:23.253607 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:45999
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:45999,127.10.242.61:38181,127.10.242.60:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:23.587944 16206 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:23.588282 16206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:23.588392 16206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:23.588505 16206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:23.595544 16206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:23.595628 16206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:23.595716 16206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:23.595782 16206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:23.595866 16206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:23.605316 16206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:45999,127.10.242.61:38181,127.10.242.60:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:45999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:23.606559 16206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:23.607901 16206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:23.619109 16219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:23.619206 16214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:23.619206 16217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:23.620271 16206 server_base.cc:1034] running on GCE node
I20260810 06:49:23.620997 16206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:23.622280 16206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:23.623499 16206 hybrid_clock.cc:648] HybridClock initialized: now 1786344563623441 us; error 28 us; skew 500 ppm
I20260810 06:49:23.623869 16206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:23 dist-test-slave-zj8q krb5kdc[16188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344563, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:23.626911 16206 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:49:23.628288 16206 webserver.cc:460] Webserver started at http://127.10.242.62:43251/ using document root <none> and password file <none>
I20260810 06:49:23.628954 16206 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:23.629052 16206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:23.629376 16206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:23.632326 16206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "4e376f8332e2467e880d02c68508a5c2"
format_stamp: "Formatted at 2026-08-10 06:49:23 on dist-test-slave-zj8q"
server_key: "e2fe4c30b807bfb38174fcec15dd7f07"
server_key_iv: "db05cfab46c63c443217e21ac627f5ad"
server_key_version: "encryptionkey@0"
I20260810 06:49:23.633108 16206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "4e376f8332e2467e880d02c68508a5c2"
format_stamp: "Formatted at 2026-08-10 06:49:23 on dist-test-slave-zj8q"
server_key: "e2fe4c30b807bfb38174fcec15dd7f07"
server_key_iv: "db05cfab46c63c443217e21ac627f5ad"
server_key_version: "encryptionkey@0"
I20260810 06:49:23.638063 16206 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:49:23.641571 16229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:23.642848 16206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 06:49:23.643014 16206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "4e376f8332e2467e880d02c68508a5c2"
format_stamp: "Formatted at 2026-08-10 06:49:23 on dist-test-slave-zj8q"
server_key: "e2fe4c30b807bfb38174fcec15dd7f07"
server_key_iv: "db05cfab46c63c443217e21ac627f5ad"
server_key_version: "encryptionkey@0"
I20260810 06:49:23.643231 16206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:23.655711 16206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:23.659574 16206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:23.659823 16206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:23.677049 16206 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:45999
I20260810 06:49:23.677068 16317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:45999 every 8 connection(s)
I20260810 06:49:23.678398 16206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:23.681265 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16206
I20260810 06:49:23.681485 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:23.682353 11208 external_mini_cluster.cc:1428] Setting key c8d4661a922d9599ab5ed6c63ff7552d
I20260810 06:49:23.682394 16318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:23.690671 16318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:23 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344563, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:49:23.713088 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.694030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:45999 (local address 127.0.0.1:53670)
0810 06:49:23.694863 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:23.694908 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:23.696816 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:23.697616 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:23.697623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:23.698332 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:23.698672 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:23.698690 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:23.700833 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:23.700838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:23.701893 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:23.701906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:23.703069 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:23.703074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:23.703214 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:23.703781 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:23.703804 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:23.705824 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:23.709403 (+ 3579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:23.709412 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:23.709421 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:23.709781 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:23.710239 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:23.710243 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:23.710245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:23.710390 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:23.711300 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:23.711305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:23.711477 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 06:49:23.711666 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 06:49:23.712017 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":322,"threads_started":1}
I20260810 06:49:23.713156 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.694030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53670 (local address 127.10.242.62:45999)
0810 06:49:23.695657 (+ 1627us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:23.695665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:23.695718 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:49:23.697167 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:23.697173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:23.697384 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:23.697569 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:23.698881 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:23.700665 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:23.702047 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:23.702865 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:23.703044 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:23.706052 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:23.706062 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:23.706085 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:23.706119 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:23.709285 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:23.709913 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:23.709921 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:23.709924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:23.710110 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:23.710487 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:23.710491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:23.710495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:23.711168 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:23.711354 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:23.711845 (+ 491us) server_negotiation.cc:299] Negotiation successful
0810 06:49:23.712206 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":1419,"thread_start_us":1166,"threads_started":1}
I20260810 06:49:23.716055 16318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.242.61@KRBTEST.COM; defaulting to no policy
I20260810 06:49:23.716964 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.716582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51216)
0810 06:49:23.716751 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:23.716854 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 06:49:23.717159 16233 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
I20260810 06:49:23.718612 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.718286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51222)
0810 06:49:23.718418 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:23.718478 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:49:23.718809 16318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38181: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab.
Entry for principal kudu/127.10.242.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab.
I20260810 06:49:23.751312 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.61:38181
--webserver_interface=127.10.242.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab
--principal=kudu/127.10.242.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:45999,127.10.242.61:38181,127.10.242.60:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:49:23.768100 16318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } attempt: 1
I20260810 06:49:23.769215 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.768783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51226)
0810 06:49:23.768965 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:23.769071 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 06:49:23.770123 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.769781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51242)
0810 06:49:23.769923 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:23.769990 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:49:23.770351 16318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38181: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
I20260810 06:49:23.838644 16318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } attempt: 2
I20260810 06:49:23.839869 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.839372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51244)
0810 06:49:23.839623 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:23.839720 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 06:49:23.841006 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.840614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51254)
0810 06:49:23.840785 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:23.840874 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 06:49:23.841218 16318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38181: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
I20260810 06:49:23.932487 16318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } attempt: 3
I20260810 06:49:23.933679 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.933227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51260)
0810 06:49:23.933436 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:23.933531 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 06:49:23.934726 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:23.934386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51262)
0810 06:49:23.934506 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:23.934584 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:49:23.934926 16318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38181: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
I20260810 06:49:24.078143 16318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } attempt: 4
I20260810 06:49:24.079537 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.078823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51276)
0810 06:49:24.079235 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.079360 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260810 06:49:24.080740 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.080331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51290)
0810 06:49:24.080479 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.080583 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:49:24.080940 16318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38181: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38181: connect: Connection refused (error 111)
W20260810 06:49:24.086791 16328 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:24.087070 16328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:24.087141 16328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:24.087221 16328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:24.094260 16328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:24.094339 16328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:24.094401 16328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:24.094451 16328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:24.094512 16328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:24.104302 16328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:45999,127.10.242.61:38181,127.10.242.60:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab
--principal=kudu/127.10.242.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.61:38181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.242.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:24.105568 16328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:24.106886 16328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:24.118455 16344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:24.118485 16350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:24.118734 16342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:24.119315 16328 server_base.cc:1034] running on GCE node
I20260810 06:49:24.120078 16328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:24.121404 16328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:24.122645 16328 hybrid_clock.cc:648] HybridClock initialized: now 1786344564122574 us; error 39 us; skew 500 ppm
I20260810 06:49:24.122968 16328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:24.126042 16328 init.cc:377] Logged in from keytab as kudu/127.10.242.61@KRBTEST.COM (short username kudu)
I20260810 06:49:24.127473 16328 webserver.cc:460] Webserver started at http://127.10.242.61:41471/ using document root <none> and password file <none>
I20260810 06:49:24.128149 16328 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:24.128255 16328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:24.128566 16328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:24.131582 16328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/data/instance:
uuid: "7b32d428336c47019ef999f868bd4290"
format_stamp: "Formatted at 2026-08-10 06:49:24 on dist-test-slave-zj8q"
server_key: "72218339820b9b538596d0723ef93feb"
server_key_iv: "827637c0ee028eec964f0dde937f3b21"
server_key_version: "encryptionkey@0"
I20260810 06:49:24.132390 16328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/wal/instance:
uuid: "7b32d428336c47019ef999f868bd4290"
format_stamp: "Formatted at 2026-08-10 06:49:24 on dist-test-slave-zj8q"
server_key: "72218339820b9b538596d0723ef93feb"
server_key_iv: "827637c0ee028eec964f0dde937f3b21"
server_key_version: "encryptionkey@0"
I20260810 06:49:24.137331 16328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:49:24.140697 16362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:24.142022 16328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:49:24.142204 16328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/wal
uuid: "7b32d428336c47019ef999f868bd4290"
format_stamp: "Formatted at 2026-08-10 06:49:24 on dist-test-slave-zj8q"
server_key: "72218339820b9b538596d0723ef93feb"
server_key_iv: "827637c0ee028eec964f0dde937f3b21"
server_key_version: "encryptionkey@0"
I20260810 06:49:24.142416 16328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:24.171820 16328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:24.175771 16328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:24.176035 16328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:24.193418 16328 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.61:38181
I20260810 06:49:24.193444 16446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.61:38181 every 8 connection(s)
I20260810 06:49:24.194736 16328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
I20260810 06:49:24.198318 16447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:24.199456 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16328
I20260810 06:49:24.199615 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-1/wal/instance
I20260810 06:49:24.200073 11208 external_mini_cluster.cc:1428] Setting key 580ba913a821b179afbcfa5814d315c1
I20260810 06:49:24.204778 16447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:49:24.226578 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.207248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53678 (local address 127.10.242.62:45999)
0810 06:49:24.207453 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.207459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.208567 (+ 1108us) server_negotiation.cc:407] Connection header received
0810 06:49:24.210960 (+ 2393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.210966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.211049 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.211147 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.213092 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.213932 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.216085 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.216673 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.216807 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.220361 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.220370 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.220375 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.220422 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.223545 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.224246 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.224251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.224254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.224397 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.224783 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.224789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.224791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.225179 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.225315 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.226016 (+ 701us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.226269 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:49:24.227301 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.207642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:45999 (local address 127.0.0.1:53678)
0810 06:49:24.208374 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.208426 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:24.210660 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:24.211389 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:24.211399 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:24.212059 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:24.212918 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.212942 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.214093 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.214098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.215920 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.215935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.216865 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.216870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.217064 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.217742 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:24.217763 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:24.220106 (+ 2343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:24.223680 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.223687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.223698 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.224106 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.224490 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.224493 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.224495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.224664 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.225295 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:24.225301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:24.225607 (+ 306us) client_negotiation.cc:769] Sending connection context
0810 06:49:24.225829 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 06:49:24.226241 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":286,"threads_started":1}
I20260810 06:49:24.228966 16447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.242.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.60@KRBTEST.COM" created.
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
I20260810 06:49:24.243351 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.229763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51306)
0810 06:49:24.229911 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.229927 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:24.230008 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:24.230797 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:24.230801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:24.231269 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:24.231442 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.231454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.232865 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.232871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.234251 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.234269 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.235090 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.235095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.235239 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.235746 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:24.235763 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:24.237793 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:24.241215 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.241218 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.241221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.241558 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.241913 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.241916 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.241919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.242048 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.242697 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:24.242700 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:24.242812 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 06:49:24.242917 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 06:49:24.243099 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 06:49:24.243685 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.229848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51306 (local address 127.10.242.61:38181)
0810 06:49:24.230415 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.230421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.230445 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:49:24.230484 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.230490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.230659 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.230791 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.231627 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.232716 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.234392 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.234953 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.235054 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.237995 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.238009 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.238025 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.238059 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.241098 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.241681 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.241690 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.241693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.241823 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.242139 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.242143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.242145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.242574 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.242780 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.243072 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.243305 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":287,"threads_started":1}
I20260810 06:49:24.245945 16447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:24.247191 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.246675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36258)
0810 06:49:24.246844 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.247028 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 06:49:24.247463 16364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
I20260810 06:49:24.248903 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.248595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36268)
0810 06:49:24.248727 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.248806 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:49:24.249091 16447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:36635: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab.
Entry for principal kudu/127.10.242.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab.
I20260810 06:49:24.270628 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.60:36635
--webserver_interface=127.10.242.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab
--principal=kudu/127.10.242.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:44819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:45999,127.10.242.61:38181,127.10.242.60:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:49:24.298388 16447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } attempt: 1
I20260810 06:49:24.299690 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.299141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36284)
0810 06:49:24.299365 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.299498 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 06:49:24.300832 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.300421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36292)
0810 06:49:24.300591 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.300691 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:49:24.301062 16447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:36635: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
I20260810 06:49:24.369300 16447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } attempt: 2
I20260810 06:49:24.370581 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.369985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36300)
0810 06:49:24.370253 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.370379 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 06:49:24.371575 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.371212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36316)
0810 06:49:24.371352 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.371438 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:49:24.371819 16447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:36635: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
I20260810 06:49:24.380201 16318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } attempt: 5
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344563, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
I20260810 06:49:24.393828 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.381060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.10.242.61:38181)
0810 06:49:24.381204 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.381208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.381228 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:49:24.381310 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.381314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.381389 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.381465 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.382454 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.383162 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.384806 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.385253 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.385342 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.388966 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.388971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.388974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.389006 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.391418 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.392173 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.392176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.392178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.392294 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.392672 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.392681 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.392683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.392940 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.393044 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.393444 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.393601 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:49:24.393996 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.380839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51322)
0810 06:49:24.381036 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.381055 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:24.381198 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:24.381524 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:24.381528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:24.382020 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:24.382308 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.382323 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.383304 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.383310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.384668 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.384683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.385394 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.385399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.385634 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.386253 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:24.386278 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:24.388784 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:24.391549 (+ 2765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.391555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.391559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.392056 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.392398 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.392402 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.392406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.392580 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.393047 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:24.393052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:24.393229 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 06:49:24.393382 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 06:49:24.393661 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 06:49:24.395113 16318 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:24.396298 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.395859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36318)
0810 06:49:24.396030 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.396133 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 06:49:24.397172 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.396837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36324)
0810 06:49:24.396960 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.397052 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 06:49:24.397356 16318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:36635: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
I20260810 06:49:24.448581 16318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } attempt: 1
I20260810 06:49:24.449786 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.449240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36332)
0810 06:49:24.449450 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.449564 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:49:24.450953 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.450568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36334)
0810 06:49:24.450724 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.450816 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 06:49:24.451201 16318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:36635: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
I20260810 06:49:24.463042 16447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } attempt: 3
I20260810 06:49:24.464191 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.463664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36336)
0810 06:49:24.463892 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.463997 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 06:49:24.465301 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.464824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36344)
0810 06:49:24.465057 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.465165 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260810 06:49:24.465541 16447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:36635: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
I20260810 06:49:24.519459 16318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } attempt: 2
I20260810 06:49:24.520737 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.520148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36354)
0810 06:49:24.520394 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.520526 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 06:49:24.521863 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.521449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36356)
0810 06:49:24.521611 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.521717 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 06:49:24.522105 16318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:36635: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
W20260810 06:49:24.607025 16460 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:24.607348 16460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:24.607448 16460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:24.607554 16460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 06:49:24.608750 16447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } attempt: 4
I20260810 06:49:24.609948 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.609393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36370)
0810 06:49:24.609639 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.609749 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:49:24.610946 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.610603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36376)
0810 06:49:24.610740 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.610822 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:49:24.611140 16447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:36635: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
W20260810 06:49:24.615370 16460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:24.615455 16460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:24.615547 16460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:24.615614 16460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:24.615705 16460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:24.625283 16460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:44819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:45999,127.10.242.61:38181,127.10.242.60:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab
--principal=kudu/127.10.242.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.60:36635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.242.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:24.626596 16460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:24.627895 16460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:49:24.628355 16318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } attempt: 3
I20260810 06:49:24.629526 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.628970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36386)
0810 06:49:24.629215 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.629316 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 06:49:24.630520 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.630102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36398)
0810 06:49:24.630312 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.630400 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260810 06:49:24.630720 16318 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:36635: Network error: Client connection negotiation failed: client connection to 127.10.242.60:36635: connect: Connection refused (error 111)
W20260810 06:49:24.639842 16465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:24.639966 16467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:24.639879 16469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:24.640954 16460 server_base.cc:1034] running on GCE node
I20260810 06:49:24.641678 16460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:24.643137 16460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:24.644351 16460 hybrid_clock.cc:648] HybridClock initialized: now 1786344564644282 us; error 34 us; skew 500 ppm
I20260810 06:49:24.644675 16460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:24.647863 16460 init.cc:377] Logged in from keytab as kudu/127.10.242.60@KRBTEST.COM (short username kudu)
I20260810 06:49:24.649307 16460 webserver.cc:460] Webserver started at http://127.10.242.60:36247/ using document root <none> and password file <none>
I20260810 06:49:24.649955 16460 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:24.650048 16460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:24.650381 16460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:24.653267 16460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/data/instance:
uuid: "0ee53432b0434b9498cb56e2d9c9db35"
format_stamp: "Formatted at 2026-08-10 06:49:24 on dist-test-slave-zj8q"
server_key: "61f9deaaba6beefae94dff7cc21ed668"
server_key_iv: "cdc232bd6d9b1f4419cb04fc873624c4"
server_key_version: "encryptionkey@0"
I20260810 06:49:24.654034 16460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/wal/instance:
uuid: "0ee53432b0434b9498cb56e2d9c9db35"
format_stamp: "Formatted at 2026-08-10 06:49:24 on dist-test-slave-zj8q"
server_key: "61f9deaaba6beefae94dff7cc21ed668"
server_key_iv: "cdc232bd6d9b1f4419cb04fc873624c4"
server_key_version: "encryptionkey@0"
I20260810 06:49:24.659175 16460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:49:24.662492 16486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:24.663734 16460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:49:24.663868 16460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/wal
uuid: "0ee53432b0434b9498cb56e2d9c9db35"
format_stamp: "Formatted at 2026-08-10 06:49:24 on dist-test-slave-zj8q"
server_key: "61f9deaaba6beefae94dff7cc21ed668"
server_key_iv: "cdc232bd6d9b1f4419cb04fc873624c4"
server_key_version: "encryptionkey@0"
I20260810 06:49:24.664039 16460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:24.698235 16460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:24.702240 16460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:24.702459 16460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:24.718676 16460 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.60:36635
I20260810 06:49:24.718699 16564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.60:36635 every 8 connection(s)
I20260810 06:49:24.719980 16460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
I20260810 06:49:24.720257 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16460
I20260810 06:49:24.720444 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344206489568-11208-0/minicluster-data/master-2/wal/instance
I20260810 06:49:24.721089 11208 external_mini_cluster.cc:1428] Setting key 4bd3f4809041c4d0c367d556e834fc42
I20260810 06:49:24.724079 16568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:24.732333 16568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:49:24.744545 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.722856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36404 (local address 127.10.242.60:36635)
0810 06:49:24.723579 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.723591 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.723633 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:49:24.725266 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.725288 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.725642 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.725947 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.727920 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.729262 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.731096 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.731741 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.731873 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.736985 (+ 5112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.736996 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.737008 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.737082 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.740438 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.741122 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.741128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.741134 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.741285 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.741688 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.741692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.741695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.742386 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.742703 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.743097 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.743497 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":326,"threads_started":1}
I20260810 06:49:24.750697 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.735126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:45999 (local address 127.0.0.1:53690)
0810 06:49:24.735851 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.735907 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:24.736062 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:24.736768 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:24.736774 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:24.737694 (+ 920us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:24.738045 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.738059 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.739257 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.739266 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.740803 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.740814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.741636 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.741642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.741826 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.742439 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:24.742460 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:24.744705 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:24.748267 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.748278 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.748284 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.748652 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.749002 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.749007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.749011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.749153 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.749661 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:24.749668 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:24.749826 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 06:49:24.749978 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 06:49:24.750311 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":288,"threads_started":1}
I20260810 06:49:24.750730 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.735199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.10.242.62:45999)
0810 06:49:24.736397 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.736403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.736424 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:49:24.736487 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.736493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.736570 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.736698 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.738218 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.739087 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.740939 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.741421 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.741547 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.744901 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.744908 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.744913 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.744951 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.748145 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.748762 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.748768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.748771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.748900 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.749254 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.749259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.749263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.749538 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.749682 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.750111 (+ 429us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.750331 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1062,"thread_start_us":959,"threads_started":1}
I20260810 06:49:24.751935 16568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
I20260810 06:49:24.762037 16581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.747618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53694 (local address 127.10.242.62:45999)
0810 06:49:24.748223 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.748227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.748246 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:49:24.748286 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.748289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.748339 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.748429 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.749582 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.750452 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.751808 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.752259 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.752383 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.756772 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.756778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.756782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.756815 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.759418 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.760010 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.760015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.760018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.760136 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.760524 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.760528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.760532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.760883 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.761153 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.761441 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.761663 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":261,"threads_started":1}
I20260810 06:49:24.765056 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.752608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38181 (local address 127.0.0.1:51324)
0810 06:49:24.752756 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.752772 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:24.752860 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:24.753181 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:24.753184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:24.753486 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:24.753611 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.753619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.754929 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.754933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.756052 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.756064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.756738 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.756743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.756874 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.757345 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:24.757361 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:24.759381 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:24.762818 (+ 3437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.762823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.762826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.763227 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.763591 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.763605 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.763608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.763785 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.764342 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:24.764347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:24.764510 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 06:49:24.764649 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 06:49:24.764836 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:49:24.765337 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.752628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51324 (local address 127.10.242.61:38181)
0810 06:49:24.752804 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.752810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.752830 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:49:24.752971 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.752976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.753054 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.753155 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.753764 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.754812 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.756190 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.756586 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.756698 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.759564 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.759570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.759575 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.759610 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.762691 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.763323 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.763327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.763329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.763473 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.763921 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.763927 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.763930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.764210 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.764347 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.764768 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.764985 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:49:24.766254 16568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
I20260810 06:49:24.779184 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.763579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.10.242.61:38181)
0810 06:49:24.764113 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.764117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.764143 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:49:24.764179 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.764182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.764229 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.764321 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.765915 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.766897 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.768278 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.768920 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.769035 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.773557 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.773564 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.773567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.773620 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.776657 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.777411 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.777415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.777417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.777533 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.777922 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.777925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.777927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.778230 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.778487 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.778693 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.778849 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":236,"threads_started":1}
I20260810 06:49:24.779577 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.767083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36408)
0810 06:49:24.767224 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.767241 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:24.767326 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:24.767704 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:24.767708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:24.768100 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:24.768230 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.768238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.769417 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.769423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.770715 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.770731 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.771410 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.771417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.771624 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.772093 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:24.772109 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:24.773986 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:24.777106 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.777111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.777114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.777580 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.777925 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.777930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.777933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.778068 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.778676 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:24.778681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:24.778826 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 06:49:24.778996 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 06:49:24.779287 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:49:24.779942 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.767298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.10.242.60:36635)
0810 06:49:24.767454 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.767460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.767477 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:49:24.767521 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.767526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.767590 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.767694 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.768368 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.769291 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.770850 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.771251 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.771338 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.774181 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.774187 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.774191 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.774226 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.776965 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.777691 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.777695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.777697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.777807 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.778202 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.778207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.778210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.778518 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.778645 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.779119 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.779593 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":203,"server-negotiator.queue_time_us":47}
I20260810 06:49:24.784183 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:24.785035 16568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35: Bootstrap starting.
I20260810 06:49:24.788641 16568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:24.789942 16568 log.cc:824] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:24.786774 16278 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4e376f8332e2467e880d02c68508a5c2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53694
I20260810 06:49:24.792953 16568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35: No bootstrap required, opened a new log
I20260810 06:49:24.803319 16568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } }
I20260810 06:49:24.803699 16568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ee53432b0434b9498cb56e2d9c9db35, State: Initialized, Role: FOLLOWER
I20260810 06:49:24.804189 16568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } }
I20260810 06:49:24.805051 16585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:24.805361 16585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:24.806289 16568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:24.807931 16318 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } attempt: 4
W20260810 06:49:24.816540 16608 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:49:24.816758 16608 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344563, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
I20260810 06:49:24.824987 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.808579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36422)
0810 06:49:24.808785 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.808804 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:24.808930 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:24.809357 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:24.809362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:24.809714 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:24.809879 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.809889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.811415 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.811420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.812901 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.812919 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.814190 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.814197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.814416 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.815012 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:24.815034 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:24.817773 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:24.821931 (+ 4158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.821936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.821940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.822469 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.822935 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.822940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.822944 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.823106 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.823955 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:24.823960 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:24.824166 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 06:49:24.824317 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 06:49:24.824599 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 06:49:24.825980 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.808593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36422 (local address 127.10.242.60:36635)
0810 06:49:24.808777 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.808784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.808878 (+ 94us) server_negotiation.cc:407] Connection header received
0810 06:49:24.809101 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.809107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.809201 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.809306 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.810049 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.811285 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.813059 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.813779 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.813912 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.817989 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.817996 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.818001 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.818037 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.821771 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.822611 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.822617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.822621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.822780 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.823257 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.823262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.823269 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.823729 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.823889 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.825401 (+ 1512us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.825649 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:49:24.815191 16460 master_runner.cc:186] Error getting master registration for 127.10.242.62:45999: OK, instance_id { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" instance_seqno: 1786344563672152 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:49:24.826525 16460 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:49:24.830389 16318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2: Bootstrap starting.
I20260810 06:49:24.833336 16318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:24.834630 16318 log.cc:824] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:24.837201 16318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2: No bootstrap required, opened a new log
I20260810 06:49:24.842025 16318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } }
I20260810 06:49:24.842358 16318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e376f8332e2467e880d02c68508a5c2, State: Initialized, Role: FOLLOWER
I20260810 06:49:24.842756 16318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } }
I20260810 06:49:24.843590 16613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:24.843997 16613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:24.844666 16318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:24.844816 16278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:49:24.845013 16278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:24.848728 16278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } }
I20260810 06:49:24.851647 16524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e376f8332e2467e880d02c68508a5c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0ee53432b0434b9498cb56e2d9c9db35"
I20260810 06:49:24.852140 16524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:24.855502 16524 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e376f8332e2467e880d02c68508a5c2 in term 1.
I20260810 06:49:24.856096 16231 leader_election.cc:302] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ee53432b0434b9498cb56e2d9c9db35, 4e376f8332e2467e880d02c68508a5c2; no voters:
I20260810 06:49:24.856591 16613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:24.862618 16278 leader_election.cc:288] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [CANDIDATE]: Term 1 election: Requested vote from peers 7b32d428336c47019ef999f868bd4290 (127.10.242.61:38181), 0ee53432b0434b9498cb56e2d9c9db35 (127.10.242.60:36635)
I20260810 06:49:24.854076 16206 master_runner.cc:186] Error getting master registration for 127.10.242.61:38181: OK, instance_id { permanent_uuid: "7b32d428336c47019ef999f868bd4290" instance_seqno: 1786344564188443 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:49:24.865367 16206 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 06:49:24.868079 16633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 06:49:24.868132 16613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [term 1 LEADER]: Becoming Leader. State: Replica: 4e376f8332e2467e880d02c68508a5c2, State: Running, Role: LEADER
W20260810 06:49:24.868268 16633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:24.868805 16613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } }
I20260810 06:49:24.871632 16637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e376f8332e2467e880d02c68508a5c2. Latest consensus state: current_term: 1 leader_uuid: "4e376f8332e2467e880d02c68508a5c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:24.872071 16637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:24.878943 16641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:24.881079 16641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:24.891983 16524 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [term 1 FOLLOWER]: Refusing update from remote peer 4e376f8332e2467e880d02c68508a5c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:24.893117 16637 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:24.910331 16447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } attempt: 5
W20260810 06:49:24.912437 16238 tablet.cc:2369] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:24.917965 16585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e376f8332e2467e880d02c68508a5c2. Latest consensus state: current_term: 1 leader_uuid: "4e376f8332e2467e880d02c68508a5c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:24.918344 16585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:24.924009 16585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e376f8332e2467e880d02c68508a5c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:24.924255 16585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:24.934800 16637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e376f8332e2467e880d02c68508a5c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:24.935076 16637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:24.937516 16641 catalog_manager.cc:1347] Generated new cluster ID: 0a767d5007794f2e92b18f0ef5a7bc97
I20260810 06:49:24.937624 16641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 06:49:24 dist-test-slave-zj8q krb5kdc[16188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
I20260810 06:49:24.958899 16641 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:24.960662 16641 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:24.967284 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.910873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:36635 (local address 127.0.0.1:36442)
0810 06:49:24.911110 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:24.911128 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:24.911241 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:24.931137 (+ 19896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:24.931143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:24.931670 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:24.931887 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.931901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.937088 (+ 5187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.937094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.938468 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:24.938486 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.939687 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.939693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:24.939892 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.940543 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:24.940568 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:24.943294 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:24.952739 (+ 9445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.952744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.952748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.953309 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.955504 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:24.955509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:24.955512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:24.955708 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:24.966260 (+ 10552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:24.966266 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:24.966435 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 06:49:24.966591 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 06:49:24.966902 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260810 06:49:24.968181 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.904220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.10.242.60:36635)
0810 06:49:24.904381 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.904386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.935378 (+ 30992us) server_negotiation.cc:407] Connection header received
0810 06:49:24.935812 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.935817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.935890 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.935955 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.941711 (+ 5756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.942882 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.949721 (+ 6839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.950477 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.950582 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.956571 (+ 5989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.956578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.956583 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.956620 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.959880 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.964678 (+ 4798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.964684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.964687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.964859 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.966311 (+ 1452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.966316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.966320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.966655 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.966878 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.967539 (+ 661us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.967775 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:49:24.968351 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.911506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36442 (local address 127.10.242.60:36635)
0810 06:49:24.930786 (+ 19280us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.930793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.930817 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:49:24.930875 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.930880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.930957 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.931069 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.932633 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.933870 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.938692 (+ 4822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.939291 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.940189 (+ 898us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.943812 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.943818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.943823 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.943859 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.952620 (+ 8761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.955212 (+ 2592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.955217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.955221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.955376 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.955853 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.955857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.955859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.956219 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.967514 (+ 11295us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.967648 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.967920 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":19155,"thread_start_us":19748,"threads_started":1}
I20260810 06:49:24.975618 16447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290: Bootstrap starting.
I20260810 06:49:24.979684 16447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:24.975715 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.899019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.10.242.61:38181)
0810 06:49:24.899226 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.899233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.932364 (+ 33131us) server_negotiation.cc:407] Connection header received
0810 06:49:24.932576 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.932581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.932655 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.932746 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.954046 (+ 21300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.955210 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.961080 (+ 5870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.961721 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.961815 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.967276 (+ 5461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.967283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.967288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.967323 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.971051 (+ 3728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.972429 (+ 1378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.972435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.972442 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.972590 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.973326 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.973332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.973335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.973668 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.973885 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.974882 (+ 997us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.975107 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:49:24.982458 16447 log.cc:824] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:24.986414 16581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:24.899017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.10.242.62:45999)
0810 06:49:24.899262 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:24.899269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:24.931898 (+ 32629us) server_negotiation.cc:407] Connection header received
0810 06:49:24.934568 (+ 2670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:24.934573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:24.934664 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:24.934741 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:24.940402 (+ 5661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.941425 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.953322 (+ 11897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:24.954010 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:24.954127 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:24.965425 (+ 11298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:24.965433 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:24.965437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:24.965477 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:24.970440 (+ 4963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.977153 (+ 6713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.977159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.977163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.977479 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:24.978345 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:24.978353 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:24.978357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:24.979163 (+ 806us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:24.984032 (+ 4869us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:24.984593 (+ 561us) server_negotiation.cc:299] Negotiation successful
0810 06:49:24.985680 (+ 1087us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:49:24.987241 16641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2: Generated new TSK 0
I20260810 06:49:24.989077 16641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:24.998678 16447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290: No bootstrap required, opened a new log
I20260810 06:49:25.010140 16447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } }
I20260810 06:49:25.010722 16447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b32d428336c47019ef999f868bd4290, State: Initialized, Role: FOLLOWER
I20260810 06:49:25.011317 16447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } }
I20260810 06:49:25.020224 16447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:25.020437 16396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4e376f8332e2467e880d02c68508a5c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7b32d428336c47019ef999f868bd4290"
I20260810 06:49:25.022359 16393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:25.025771 16393 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [term 1 FOLLOWER]: Refusing update from remote peer 4e376f8332e2467e880d02c68508a5c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:25.029876 16396 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e376f8332e2467e880d02c68508a5c2 in term 1.
I20260810 06:49:25.030823 16662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:25.031268 16662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:25.031565 16613 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:25.078631 16662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e376f8332e2467e880d02c68508a5c2. Latest consensus state: current_term: 1 leader_uuid: "4e376f8332e2467e880d02c68508a5c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:25.078985 16662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:25.079427 16681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e376f8332e2467e880d02c68508a5c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:25.079921 16681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:25.097961 16696 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260810 06:49:25.116995 16581 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.108948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.10.242.62:45999)
0810 06:49:25.109136 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.109142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.114072 (+ 4930us) server_negotiation.cc:407] Connection header received
0810 06:49:25.115327 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.115334 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.115442 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.115515 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.116722 (+ 1207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53706: BlockingRecv error: recv got EOF from 127.0.0.1:53706 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 06:49:25.119993 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.110810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.10.242.61:38181)
0810 06:49:25.110966 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.110973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.118537 (+ 7564us) server_negotiation.cc:407] Connection header received
0810 06:49:25.118699 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.118705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.118784 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.118868 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.119778 (+ 910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51350: BlockingRecv error: recv got EOF from 127.0.0.1:51350 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 06:49:25.120971 16654 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.111353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36456 (local address 127.10.242.60:36635)
0810 06:49:25.111555 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.111561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.119145 (+ 7584us) server_negotiation.cc:407] Connection header received
0810 06:49:25.119320 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.119325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.119406 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.119500 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.120714 (+ 1214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36456: BlockingRecv error: recv got EOF from 127.0.0.1:36456 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 06:49:25.131847 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.129614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.10.242.61:38181)
0810 06:49:25.129777 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.129782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.130267 (+ 485us) server_negotiation.cc:407] Connection header received
0810 06:49:25.130565 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.130571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.130682 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.130748 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.131647 (+ 899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51354: BlockingRecv error: recv got EOF from 127.0.0.1:51354 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 06:49:25.139211 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.137087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.10.242.61:38181)
0810 06:49:25.137260 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.137265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.137690 (+ 425us) server_negotiation.cc:407] Connection header received
0810 06:49:25.137933 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.137938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.138015 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.138075 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.138965 (+ 890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51360: BlockingRecv error: recv got EOF from 127.0.0.1:51360 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 06:49:25.148309 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.146188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.10.242.61:38181)
0810 06:49:25.146362 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.146368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.146842 (+ 474us) server_negotiation.cc:407] Connection header received
0810 06:49:25.147041 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.147046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.147138 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.147225 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.148082 (+ 857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51376: BlockingRecv error: recv got EOF from 127.0.0.1:51376 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 06:49:25.164114 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.159446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51390 (local address 127.10.242.61:38181)
0810 06:49:25.159639 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.159645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.162420 (+ 2775us) server_negotiation.cc:407] Connection header received
0810 06:49:25.162585 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.162590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.162670 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.162759 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.163891 (+ 1132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51390: BlockingRecv error: recv got EOF from 127.0.0.1:51390 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260810 06:49:25.177992 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.176020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.10.242.61:38181)
0810 06:49:25.176205 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.176211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.176704 (+ 493us) server_negotiation.cc:407] Connection header received
0810 06:49:25.176881 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.176887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.176971 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.177065 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.177802 (+ 737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51396: BlockingRecv error: recv got EOF from 127.0.0.1:51396 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 06:49:25.225004 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.222648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.10.242.61:38181)
0810 06:49:25.222841 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.222847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.223441 (+ 594us) server_negotiation.cc:407] Connection header received
0810 06:49:25.223657 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.223663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.223763 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.223883 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.224750 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51408: BlockingRecv error: recv got EOF from 127.0.0.1:51408 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 06:49:25.263553 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.261084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.10.242.61:38181)
0810 06:49:25.261295 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.261302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.261864 (+ 562us) server_negotiation.cc:407] Connection header received
0810 06:49:25.262110 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.262116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.262252 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.262372 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.263318 (+ 946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51420: BlockingRecv error: recv got EOF from 127.0.0.1:51420 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 06:49:25.336478 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.334121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.10.242.61:38181)
0810 06:49:25.334343 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.334350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.334972 (+ 622us) server_negotiation.cc:407] Connection header received
0810 06:49:25.335191 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.335197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.335321 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.335434 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.336235 (+ 801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51422: BlockingRecv error: recv got EOF from 127.0.0.1:51422 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260810 06:49:25.470654 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.468535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.10.242.61:38181)
0810 06:49:25.468702 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.468707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.469387 (+ 680us) server_negotiation.cc:407] Connection header received
0810 06:49:25.469574 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.469578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.469654 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.469768 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.470428 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51430: BlockingRecv error: recv got EOF from 127.0.0.1:51430 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:49:25.519663 16613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e376f8332e2467e880d02c68508a5c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e376f8332e2467e880d02c68508a5c2" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 45999 } } peers { permanent_uuid: "7b32d428336c47019ef999f868bd4290" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38181 } } peers { permanent_uuid: "0ee53432b0434b9498cb56e2d9c9db35" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 36635 } } }
I20260810 06:49:25.519984 16613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e376f8332e2467e880d02c68508a5c2 [sys.catalog]: This master's current role is: LEADER
W20260810 06:49:25.732726 16582 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:25.730654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51434 (local address 127.10.242.61:38181)
0810 06:49:25.730843 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:25.730850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:25.731335 (+ 485us) server_negotiation.cc:407] Connection header received
0810 06:49:25.731522 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:25.731526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:25.731615 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:25.731716 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:25.732473 (+ 757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51434: BlockingRecv error: recv got EOF from 127.0.0.1:51434 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:49:25.818912 16608 catalog_manager.cc:1259] Loaded cluster ID: 0a767d5007794f2e92b18f0ef5a7bc97
I20260810 06:49:25.819020 16608 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35: loading cluster ID for follower catalog manager: success
I20260810 06:49:25.821565 16608 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35: acquiring CA information for follower catalog manager: success
I20260810 06:49:25.823387 16608 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0ee53432b0434b9498cb56e2d9c9db35: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:49:26.102540 16696 catalog_manager.cc:1259] Loaded cluster ID: 0a767d5007794f2e92b18f0ef5a7bc97
I20260810 06:49:26.102681 16696 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290: loading cluster ID for follower catalog manager: success
I20260810 06:49:26.105208 16696 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290: acquiring CA information for follower catalog manager: success
I20260810 06:49:26.107131 16696 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b32d428336c47019ef999f868bd4290: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:49:26.256172 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:26.249273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.10.242.61:38181)
0810 06:49:26.249851 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:26.249858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:26.249880 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:49:26.249961 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:26.249967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:26.250237 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:26.250403 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:26.250792 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.251976 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.253254 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.253938 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.254073 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:26.254695 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:26.255463 (+ 768us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:26.255618 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:26.255721 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:49:26.255807 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":276,"threads_started":1}
I20260810 06:49:26.271488 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:26.262866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36458 (local address 127.10.242.60:36635)
0810 06:49:26.266480 (+ 3614us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:26.266484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:26.266500 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:49:26.266598 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:26.266601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:26.266749 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:26.266901 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:26.267312 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.268100 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.269377 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.269776 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.269883 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:26.270391 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:26.270894 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:26.271024 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:26.271123 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 06:49:26.271204 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3515,"thread_start_us":3424,"threads_started":1}
I20260810 06:49:26.300311 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:26.277977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53722 (local address 127.10.242.62:45999)
0810 06:49:26.287799 (+ 9822us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:26.287806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:26.293554 (+ 5748us) server_negotiation.cc:407] Connection header received
0810 06:49:26.293807 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:26.293814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:26.294051 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:26.294266 (+ 215us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:26.295076 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.296246 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.297880 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.298338 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.298450 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:26.298902 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:26.299364 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:26.299470 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:26.299929 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 06:49:26.300011 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9698,"thread_start_us":9598,"threads_started":1}
I20260810 06:49:26.302947 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:26.278183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.10.242.61:38181)
0810 06:49:26.282963 (+ 4780us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:26.282969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:26.296300 (+ 13331us) server_negotiation.cc:407] Connection header received
0810 06:49:26.296779 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:26.296784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:26.296856 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:26.296934 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:26.297595 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.298694 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.300772 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.301400 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.301512 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:26.302190 (+ 678us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:26.302321 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:26.302385 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:26.302613 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 06:49:26.302702 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4634}
I20260810 06:49:26.305361 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:26.278512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.10.242.60:36635)
0810 06:49:26.284431 (+ 5919us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:26.284437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:26.295692 (+ 11255us) server_negotiation.cc:407] Connection header received
0810 06:49:26.296456 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:26.296462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:26.296552 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:26.296610 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:26.299038 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.300026 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.301917 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:26.302468 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:26.302548 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:26.304583 (+ 2035us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:26.304713 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:26.304780 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:26.305015 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 06:49:26.305095 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5799}
I20260810 06:49:26.306128 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16206
I20260810 06:49:26.307278 16308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:26.417209 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16206
W20260810 06:49:26.452188 16487 connection.cc:535] server connection from 127.0.0.1:36422 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 06:49:26.452481 16366 connection.cc:535] server connection from 127.0.0.1:51322 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:49:26.452924 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16328
I20260810 06:49:26.454061 16435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:26.564627 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16328
I20260810 06:49:26.592247 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16460
I20260810 06:49:26.593372 16558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:26.697106 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16460
2026-08-10T06:49:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5008 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:26 dist-test-slave-zj8q krb5kdc[16831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:26 dist-test-slave-zj8q krb5kdc[16831](info): set up 2 sockets
Aug 10 06:49:26 dist-test-slave-zj8q krb5kdc[16831](info): commencing operation
krb5kdc: starting...
W20260810 06:49:27.954864 11208 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/test-user.keytab.
Aug 10 06:49:28 dist-test-slave-zj8q krb5kdc[16831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.242.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
Entry for principal kudu/127.10.242.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab.
I20260810 06:49:28.246889 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:43197
--webserver_interface=127.10.242.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:43197,127.10.242.61:38385,127.10.242.60:41367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:28.576906 16853 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:28.577273 16853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:28.577381 16853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:28.577498 16853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:28.584993 16853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:28.585088 16853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:28.585170 16853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:28.585237 16853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:28.585320 16853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:28.594975 16853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:43197,127.10.242.61:38385,127.10.242.60:41367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.62.keytab
--principal=kudu/127.10.242.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.62:43197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.242.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:28.596233 16853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:28.597513 16853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:28.608670 16861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:28.608757 16862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:28.608724 16866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:28.609766 16853 server_base.cc:1034] running on GCE node
I20260810 06:49:28.610500 16853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:28.611763 16853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:28.612969 16853 hybrid_clock.cc:648] HybridClock initialized: now 1786344568612906 us; error 29 us; skew 500 ppm
I20260810 06:49:28.613268 16853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:28 dist-test-slave-zj8q krb5kdc[16831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:28.616410 16853 init.cc:377] Logged in from keytab as kudu/127.10.242.62@KRBTEST.COM (short username kudu)
I20260810 06:49:28.617753 16853 webserver.cc:460] Webserver started at http://127.10.242.62:40227/ using document root <none> and password file <none>
I20260810 06:49:28.618462 16853 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:28.618557 16853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:28.618830 16853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:28.621985 16853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/data/instance:
uuid: "9ffa11886a4f4c64b130a9fd2df89895"
format_stamp: "Formatted at 2026-08-10 06:49:28 on dist-test-slave-zj8q"
server_key: "7022b041c6e5d9a4b35b57ee91e26d71"
server_key_iv: "1368ee0d0211432c2f4de9ad8d3d540d"
server_key_version: "encryptionkey@0"
I20260810 06:49:28.622843 16853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/wal/instance:
uuid: "9ffa11886a4f4c64b130a9fd2df89895"
format_stamp: "Formatted at 2026-08-10 06:49:28 on dist-test-slave-zj8q"
server_key: "7022b041c6e5d9a4b35b57ee91e26d71"
server_key_iv: "1368ee0d0211432c2f4de9ad8d3d540d"
server_key_version: "encryptionkey@0"
I20260810 06:49:28.628075 16853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:49:28.631708 16879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:28.633028 16853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:49:28.633194 16853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/wal
uuid: "9ffa11886a4f4c64b130a9fd2df89895"
format_stamp: "Formatted at 2026-08-10 06:49:28 on dist-test-slave-zj8q"
server_key: "7022b041c6e5d9a4b35b57ee91e26d71"
server_key_iv: "1368ee0d0211432c2f4de9ad8d3d540d"
server_key_version: "encryptionkey@0"
I20260810 06:49:28.633402 16853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:28.647223 16853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:28.650875 16853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:28.651121 16853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:28.668256 16853 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.62:43197
I20260810 06:49:28.668298 16960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.62:43197 every 8 connection(s)
I20260810 06:49:28.669516 16853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:28.672955 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16853
I20260810 06:49:28.673024 16961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:28.673161 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-0/wal/instance
I20260810 06:49:28.673604 11208 external_mini_cluster.cc:1428] Setting key 5a089a6beccff38e99717dc4bbc8475b
I20260810 06:49:28.679431 16961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:28 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:49:28.700062 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.682274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.10.242.62:43197)
0810 06:49:28.683897 (+ 1623us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:28.683908 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:28.683966 (+ 58us) server_negotiation.cc:407] Connection header received
0810 06:49:28.685350 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:28.685355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:28.685579 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:28.685755 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:28.687018 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:28.688233 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:28.689549 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:28.690213 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:28.690391 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:28.693619 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:28.693627 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:28.693637 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:28.693668 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:28.696201 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:28.696813 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:28.696819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:28.696821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:28.696954 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:28.697318 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:28.697323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:28.697325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:28.697757 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:28.697944 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:28.698473 (+ 529us) server_negotiation.cc:299] Negotiation successful
0810 06:49:28.698882 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":165,"server-negotiator.queue_time_us":1418,"thread_start_us":1222,"threads_started":1}
I20260810 06:49:28.700094 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.682274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:43197 (local address 127.0.0.1:55514)
0810 06:49:28.683038 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:28.683084 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:28.685011 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:28.685794 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:28.685801 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:28.686493 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:28.686819 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:28.686837 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:28.688370 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:28.688374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:28.689416 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:28.689429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:28.690429 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:28.690436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:28.690602 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:28.691159 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:28.691187 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:28.693422 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:28.696317 (+ 2895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:28.696328 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:28.696338 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:28.696699 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:28.697056 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:28.697061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:28.697065 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:28.697215 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:28.697883 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:28.697890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:28.698079 (+ 189us) client_negotiation.cc:769] Sending connection context
0810 06:49:28.698338 (+ 259us) client_negotiation.cc:240] Negotiation successful
0810 06:49:28.698771 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":304,"threads_started":1}
I20260810 06:49:28.703020 16961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.242.61@KRBTEST.COM; defaulting to no policy
I20260810 06:49:28.703989 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.703620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59910)
0810 06:49:28.703774 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:28.703875 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 06:49:28.704192 16886 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
I20260810 06:49:28.705582 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.705229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59926)
0810 06:49:28.705372 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:28.705435 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.61@KRBTEST.COM" created.
W20260810 06:49:28.705801 16961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38385: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab.
Entry for principal kudu/127.10.242.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab.
I20260810 06:49:28.738051 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.61:38385
--webserver_interface=127.10.242.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab
--principal=kudu/127.10.242.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:43197,127.10.242.61:38385,127.10.242.60:41367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:49:28.755138 16961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } attempt: 1
I20260810 06:49:28.756280 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.755831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59936)
0810 06:49:28.756024 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:28.756135 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 06:49:28.757296 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.756965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59948)
0810 06:49:28.757096 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:28.757166 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:49:28.757483 16961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38385: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
I20260810 06:49:28.825724 16961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } attempt: 2
I20260810 06:49:28.826967 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.826472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59958)
0810 06:49:28.826736 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:28.826828 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:49:28.827997 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.827639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59966)
0810 06:49:28.827780 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:28.827860 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 06:49:28.828186 16961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38385: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
I20260810 06:49:28.919401 16961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } attempt: 3
I20260810 06:49:28.920511 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.920103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59972)
0810 06:49:28.920284 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:28.920376 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 06:49:28.921587 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:28.921253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59982)
0810 06:49:28.921387 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:28.921455 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:49:28.921784 16961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38385: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
W20260810 06:49:29.063825 16980 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:29.064144 16980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:29.064217 16980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:29.064299 16980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 06:49:29.065065 16961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } attempt: 4
I20260810 06:49:29.066259 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.065711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59986)
0810 06:49:29.066001 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.066087 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260810 06:49:29.067250 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.066912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:59996)
0810 06:49:29.067041 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.067125 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:49:29.067418 16961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.61:38385: Network error: Client connection negotiation failed: client connection to 127.10.242.61:38385: connect: Connection refused (error 111)
W20260810 06:49:29.071822 16980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:29.071902 16980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:29.071964 16980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:29.072013 16980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:29.072072 16980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:29.081796 16980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:43197,127.10.242.61:38385,127.10.242.60:41367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.61.keytab
--principal=kudu/127.10.242.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.61:38385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.242.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:29.083027 16980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:29.084386 16980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:29.095531 16989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:29.095620 16987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:29.095839 16985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:29.096608 16980 server_base.cc:1034] running on GCE node
I20260810 06:49:29.097319 16980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:29.098634 16980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:29.099839 16980 hybrid_clock.cc:648] HybridClock initialized: now 1786344569099770 us; error 42 us; skew 500 ppm
I20260810 06:49:29.100116 16980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:29.103137 16980 init.cc:377] Logged in from keytab as kudu/127.10.242.61@KRBTEST.COM (short username kudu)
I20260810 06:49:29.104499 16980 webserver.cc:460] Webserver started at http://127.10.242.61:40763/ using document root <none> and password file <none>
I20260810 06:49:29.105115 16980 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:29.105216 16980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:29.105475 16980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:29.108352 16980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/data/instance:
uuid: "c16faafa8bba41868c48338647ff1ffd"
format_stamp: "Formatted at 2026-08-10 06:49:29 on dist-test-slave-zj8q"
server_key: "3a2df8edd5d0dc7fbac23d74d35da68f"
server_key_iv: "44fd07613e66bfe3a97f76a65895a5b1"
server_key_version: "encryptionkey@0"
I20260810 06:49:29.109074 16980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/wal/instance:
uuid: "c16faafa8bba41868c48338647ff1ffd"
format_stamp: "Formatted at 2026-08-10 06:49:29 on dist-test-slave-zj8q"
server_key: "3a2df8edd5d0dc7fbac23d74d35da68f"
server_key_iv: "44fd07613e66bfe3a97f76a65895a5b1"
server_key_version: "encryptionkey@0"
I20260810 06:49:29.113857 16980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20260810 06:49:29.117055 17001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:29.118296 16980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:49:29.118444 16980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/wal
uuid: "c16faafa8bba41868c48338647ff1ffd"
format_stamp: "Formatted at 2026-08-10 06:49:29 on dist-test-slave-zj8q"
server_key: "3a2df8edd5d0dc7fbac23d74d35da68f"
server_key_iv: "44fd07613e66bfe3a97f76a65895a5b1"
server_key_version: "encryptionkey@0"
I20260810 06:49:29.118625 16980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:29.150269 16980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:29.154176 16980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:29.154436 16980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:29.171777 16980 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.61:38385
I20260810 06:49:29.171788 17086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.61:38385 every 8 connection(s)
I20260810 06:49:29.173069 16980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/data/info.pb
I20260810 06:49:29.175258 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 16980
I20260810 06:49:29.175452 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-1/wal/instance
I20260810 06:49:29.176013 11208 external_mini_cluster.cc:1428] Setting key 1007d2c7fffaf65590e8175ef9778ca5
I20260810 06:49:29.177289 17087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:29.185474 17087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:49:29.206465 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.188584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.10.242.62:43197)
0810 06:49:29.188791 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.188796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.189886 (+ 1090us) server_negotiation.cc:407] Connection header received
0810 06:49:29.191792 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.191800 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.191888 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.192010 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.193780 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.194518 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.196522 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.196900 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.196984 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.200913 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.200918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.200921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.200952 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.203486 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.204281 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.204285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.204287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.204398 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.204815 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.204818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.204820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.205073 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.205186 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.206024 (+ 838us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.206240 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:49:29.207834 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.188972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:43197 (local address 127.0.0.1:55524)
0810 06:49:29.189724 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.189771 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:29.191501 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:29.192214 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:29.192226 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:29.192902 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:29.193614 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.193638 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.194675 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.194681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.196396 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.196411 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.197154 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.197163 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.197417 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.198062 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:29.198089 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:29.200700 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:29.203617 (+ 2917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.203626 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.203639 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.204179 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.204504 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.204508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.204511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.204728 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.205190 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:29.205198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:29.205611 (+ 413us) client_negotiation.cc:769] Sending connection context
0810 06:49:29.205928 (+ 317us) client_negotiation.cc:240] Negotiation successful
0810 06:49:29.206488 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":303,"threads_started":1}
I20260810 06:49:29.209537 17087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.242.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.242.60@KRBTEST.COM" created.
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
I20260810 06:49:29.223955 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.210298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:60006)
0810 06:49:29.210458 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.210476 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:29.210561 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:29.211458 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:29.211462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:29.211856 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:29.211993 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.212002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.213651 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.213656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.214510 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.214523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.215517 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.215521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.215685 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.216169 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:29.216186 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:29.218147 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:29.221628 (+ 3481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.221633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.221637 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.221974 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.222354 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.222358 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.222362 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.222510 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.223100 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:29.223105 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:29.223235 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 06:49:29.223340 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 06:49:29.223589 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":26}
I20260810 06:49:29.224098 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.210455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60006 (local address 127.10.242.61:38385)
0810 06:49:29.211071 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.211077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.211101 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:49:29.211144 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.211149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.211322 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.211450 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.212156 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.213499 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.214649 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.215391 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.215510 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.218399 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.218412 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.218427 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.218459 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.221525 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.222097 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.222105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.222109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.222257 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.222652 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.222657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.222661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.222978 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.223161 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.223516 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.223740 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":453,"thread_start_us":321,"threads_started":1}
I20260810 06:49:29.226462 17087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:29.227424 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.227005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35238)
0810 06:49:29.227171 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.227291 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:49:29.227697 17003 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
I20260810 06:49:29.229103 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.228805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35254)
0810 06:49:29.228930 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.229005 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 06:49:29.229321 17087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:41367: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.242.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab.
Entry for principal kudu/127.10.242.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab.
I20260810 06:49:29.247251 11208 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
/tmp/dist-test-taskcYuaB1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.60:41367
--webserver_interface=127.10.242.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab
--principal=kudu/127.10.242.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.242.20:40779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.242.62:43197,127.10.242.61:38385,127.10.242.60:41367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:49:29.278600 17087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } attempt: 1
I20260810 06:49:29.279910 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.279451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35258)
0810 06:49:29.279650 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.279759 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 06:49:29.280853 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.280514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35262)
0810 06:49:29.280674 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.280752 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 06:49:29.281036 17087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:41367: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
I20260810 06:49:29.349253 17087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } attempt: 2
I20260810 06:49:29.350466 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.349970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35264)
0810 06:49:29.350205 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.350303 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 06:49:29.351469 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.351134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35272)
0810 06:49:29.351281 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.351356 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:49:29.351670 17087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:41367: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
I20260810 06:49:29.366655 16961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } attempt: 5
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
I20260810 06:49:29.379554 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.367291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:60014)
0810 06:49:29.367492 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.367509 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:29.367647 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:29.367991 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:29.367996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:29.368393 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:29.368581 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.368591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.369579 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.369585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.370509 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.370523 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.371231 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.371237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.371450 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.371994 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:29.372012 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:29.374253 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:29.377248 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.377253 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.377258 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.377617 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.377964 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.377969 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.377973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.378106 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.378656 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:29.378661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:29.378813 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 06:49:29.378925 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:49:29.379202 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 06:49:29.379556 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.367293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60014 (local address 127.10.242.61:38385)
0810 06:49:29.367445 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.367451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.367575 (+ 124us) server_negotiation.cc:407] Connection header received
0810 06:49:29.367774 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.367778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.367856 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.367938 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.368736 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.369445 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.370654 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.371085 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.371204 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.374475 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.374483 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.374488 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.374527 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.377121 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.377730 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.377736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.377740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.377861 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.378252 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.378257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.378261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.378539 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.378681 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.379071 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.379275 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:49:29.380860 16961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:29.381882 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.381526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35278)
0810 06:49:29.381678 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.381751 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 06:49:29.382781 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.382511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35286)
0810 06:49:29.382618 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.382694 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:49:29.382977 16961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:41367: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
I20260810 06:49:29.434191 16961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } attempt: 1
I20260810 06:49:29.435293 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.434864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35298)
0810 06:49:29.435037 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.435136 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:49:29.436300 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.435982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35302)
0810 06:49:29.436131 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.436199 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 06:49:29.436506 16961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:41367: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
I20260810 06:49:29.442857 17087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } attempt: 3
I20260810 06:49:29.443719 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.443329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35310)
0810 06:49:29.443494 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.443574 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 06:49:29.444818 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.444413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35314)
0810 06:49:29.444638 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.444712 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260810 06:49:29.445031 17087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:41367: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
I20260810 06:49:29.504703 16961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } attempt: 2
I20260810 06:49:29.505863 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.505402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35324)
0810 06:49:29.505601 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.505707 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:49:29.506841 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.506536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35326)
0810 06:49:29.506658 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.506725 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:49:29.507043 16961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:41367: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
W20260810 06:49:29.577024 17095 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:49:29.577327 17095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:29.577426 17095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:29.577553 17095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:29.584596 17095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:29.584674 17095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:29.584764 17095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:29.584836 17095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:29.584919 17095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:29.588235 17087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } attempt: 4
I20260810 06:49:29.589437 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.588910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35340)
0810 06:49:29.589155 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.589252 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 06:49:29.590342 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.590028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35344)
0810 06:49:29.590170 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.590244 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:49:29.590531 17087 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:41367: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
I20260810 06:49:29.594843 17095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.242.20:40779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.242.62:43197,127.10.242.61:38385,127.10.242.60:41367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/krb5kdc/kudu_127.10.242.60.keytab
--principal=kudu/127.10.242.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.242.60:41367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.242.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:29.596021 17095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:29.597294 17095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:29.608249 17102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:29.608286 17103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:29.608316 17105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:29.608876 17095 server_base.cc:1034] running on GCE node
I20260810 06:49:29.609572 17095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:29.610886 17095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:29.612124 17095 hybrid_clock.cc:648] HybridClock initialized: now 1786344569612057 us; error 30 us; skew 500 ppm
I20260810 06:49:29.612421 17095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:49:29.613224 16961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } attempt: 3
I20260810 06:49:29.614140 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.613719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35350)
0810 06:49:29.613918 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.613994 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:29.615263 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.614857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35356)
0810 06:49:29.615072 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.615149 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260810 06:49:29.615471 16961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.242.60:41367: Network error: Client connection negotiation failed: client connection to 127.10.242.60:41367: connect: Connection refused (error 111)
I20260810 06:49:29.615784 17095 init.cc:377] Logged in from keytab as kudu/127.10.242.60@KRBTEST.COM (short username kudu)
I20260810 06:49:29.617129 17095 webserver.cc:460] Webserver started at http://127.10.242.60:44731/ using document root <none> and password file <none>
I20260810 06:49:29.617743 17095 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:29.617831 17095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:29.618086 17095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:29.620857 17095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/data/instance:
uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75"
format_stamp: "Formatted at 2026-08-10 06:49:29 on dist-test-slave-zj8q"
server_key: "e005894687ae350aa32160b519f92c75"
server_key_iv: "82f85f046c5b8d56f78af7fe1b194476"
server_key_version: "encryptionkey@0"
I20260810 06:49:29.621603 17095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/wal/instance:
uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75"
format_stamp: "Formatted at 2026-08-10 06:49:29 on dist-test-slave-zj8q"
server_key: "e005894687ae350aa32160b519f92c75"
server_key_iv: "82f85f046c5b8d56f78af7fe1b194476"
server_key_version: "encryptionkey@0"
I20260810 06:49:29.626413 17095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260810 06:49:29.629621 17117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:29.630913 17095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:29.631045 17095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/data,/tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/wal
uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75"
format_stamp: "Formatted at 2026-08-10 06:49:29 on dist-test-slave-zj8q"
server_key: "e005894687ae350aa32160b519f92c75"
server_key_iv: "82f85f046c5b8d56f78af7fe1b194476"
server_key_version: "encryptionkey@0"
I20260810 06:49:29.631218 17095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:29.655668 17095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:29.659355 17095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:29.659569 17095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:29.676744 17095 rpc_server.cc:307] RPC server started. Bound to: 127.10.242.60:41367
I20260810 06:49:29.676752 17196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.242.60:41367 every 8 connection(s)
I20260810 06:49:29.678301 17095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/data/info.pb
I20260810 06:49:29.681850 17198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:29.684252 11208 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu as pid 17095
I20260810 06:49:29.684439 11208 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcYuaB1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344206489568-11208-0/minicluster-data/master-2/wal/instance
I20260810 06:49:29.685009 11208 external_mini_cluster.cc:1428] Setting key ca2fa36cad841f20890b4a9f33d3065f
I20260810 06:49:29.689426 17198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:49:29.706874 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.692071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55532 (local address 127.10.242.62:43197)
0810 06:49:29.692232 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.692236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.692943 (+ 707us) server_negotiation.cc:407] Connection header received
0810 06:49:29.693115 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.693122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.693191 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.693272 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.694629 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.695473 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.697137 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.697552 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.697640 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.700866 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.700872 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.700876 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.700914 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.703870 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.704629 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.704632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.704635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.704778 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.705188 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.705194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.705197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.705540 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.705671 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.706329 (+ 658us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.706526 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:49:29.707604 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.692210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.62:43197 (local address 127.0.0.1:55532)
0810 06:49:29.692809 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.692847 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:29.692999 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:29.693358 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:29.693365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:29.694138 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:29.694482 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.694497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.695637 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.695641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.696982 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.696998 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.697773 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.697781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.697967 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.698543 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:29.698571 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:29.700677 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:29.703997 (+ 3320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.704009 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.704019 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.704521 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.704883 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.704886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.704888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.705079 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.705666 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:29.705674 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:29.705903 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 06:49:29.706226 (+ 323us) client_negotiation.cc:240] Negotiation successful
0810 06:49:29.706685 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":144,"spinlock_wait_cycles":13568,"thread_start_us":277,"threads_started":1}
I20260810 06:49:29.707641 17201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.686755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35372 (local address 127.10.242.60:41367)
0810 06:49:29.687538 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.687550 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.687590 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:49:29.689262 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.689285 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.689644 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.689955 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.691748 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.693554 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.695120 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.695972 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.696110 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.700433 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.700441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.700452 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.700481 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.703633 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.704341 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.704348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.704350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.704498 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.704875 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.704880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.704884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.705455 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.705780 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.706098 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.706543 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":351,"threads_started":1}
I20260810 06:49:29.709024 17198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.62@KRBTEST.COM
I20260810 06:49:29.722414 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.709691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.61:38385 (local address 127.0.0.1:60018)
0810 06:49:29.709837 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.709854 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:29.709958 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:29.710489 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:29.710494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:29.710889 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:29.711011 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.711019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.712225 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.712231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.713306 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.713317 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.714200 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.714206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.714359 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.714815 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:29.714837 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:29.716843 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:29.720009 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.720014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.720018 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.720484 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.720865 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.720869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.720873 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.721038 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.721511 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:29.721516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:29.721685 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 06:49:29.721855 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 06:49:29.722096 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:49:29.722468 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.709700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.10.242.61:38385)
0810 06:49:29.709877 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.709882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.709901 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:49:29.710099 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.710106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.710211 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.710308 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.711163 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.712098 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.713441 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.714051 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.714217 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.717039 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.717045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.717050 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.717089 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.719886 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.720581 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.720586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.720589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.720761 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.721129 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.721132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.721134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.721371 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.721481 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.721942 (+ 461us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.722101 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:49:29.723335 17198 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:49:29.724773 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.710271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55544 (local address 127.10.242.62:43197)
0810 06:49:29.710439 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.710445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.710460 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:49:29.710501 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.710505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.710569 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.710667 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.711887 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.712818 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.714365 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.714921 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.715043 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.719155 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.719159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.719162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.719190 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.722099 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.722950 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.722954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.722956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.723085 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.723439 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.723443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.723446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.723778 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.724037 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.724332 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.724520 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.60@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.242.61@KRBTEST.COM
I20260810 06:49:29.735342 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.724102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35376)
0810 06:49:29.724253 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.724270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:29.724352 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:29.724783 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:29.724786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:29.725167 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:29.725344 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.725355 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.726283 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.726288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.727552 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.727568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.728234 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.728239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.728427 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.728882 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:29.728896 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:29.730584 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:29.733130 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.733135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.733139 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.733499 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.733814 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.733819 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.733823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.733941 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.734518 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:29.734522 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:29.734648 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 06:49:29.734758 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 06:49:29.735005 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 06:49:29.735504 17201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.724323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.10.242.60:41367)
0810 06:49:29.724458 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.724463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.724480 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:49:29.724523 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.724528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.724587 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.724673 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.725468 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.726135 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.727701 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.728094 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.728201 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.730749 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.730753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.730756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.730783 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.733013 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.733604 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.733607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.733609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.733718 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.734034 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.734039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.734042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.734365 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.734507 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.734869 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.735152 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":47}
I20260810 06:49:29.739459 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.726300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.10.242.61:38385)
0810 06:49:29.726432 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.726436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.726497 (+ 61us) server_negotiation.cc:407] Connection header received
0810 06:49:29.726642 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.726645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.726706 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.726774 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.728146 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.728941 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.730080 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.730542 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.730658 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.734323 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.734330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.734334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.734368 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.737010 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.737809 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.737813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.737815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.737941 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.738333 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.738336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.738338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.738579 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.738768 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.739065 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.739235 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:49:29.740932 17198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75: Bootstrap starting.
I20260810 06:49:29.743827 11208 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:49:29.744318 17198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:29.745299 17198 log.cc:824] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:29.747991 17198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75: No bootstrap required, opened a new log
I20260810 06:49:29.746109 16924 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9ffa11886a4f4c64b130a9fd2df89895"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55544
I20260810 06:49:29.754666 17198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } }
I20260810 06:49:29.754967 17198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e439a4a9b94cfcbfc1f2bbaee56c75, State: Initialized, Role: FOLLOWER
I20260810 06:49:29.755340 17198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } }
I20260810 06:49:29.756191 17208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:29.756556 17208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:29.757083 17198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:49:29.765713 17222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:49:29.765838 17222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:29.763515 17095 master_runner.cc:186] Error getting master registration for 127.10.242.62:43197: OK, instance_id { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" instance_seqno: 1786344568663412 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:49:29.771133 17095 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:49:29.792703 16961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } attempt: 4
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344568, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.62@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
I20260810 06:49:29.808506 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.793494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35378)
0810 06:49:29.793708 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.793731 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:29.793871 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:29.794251 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:29.794256 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:29.794759 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:29.794969 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.794982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.796429 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.796434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.797895 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.797910 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.798947 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.798953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.799165 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.799756 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:29.799774 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:29.802193 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:29.805748 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.805753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.805757 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.806284 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.806655 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.806660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.806664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.806833 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.807488 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:29.807493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:29.807695 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 06:49:29.807850 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 06:49:29.808119 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 06:49:29.808560 17201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.793532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.10.242.60:41367)
0810 06:49:29.793735 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.793742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.793829 (+ 87us) server_negotiation.cc:407] Connection header received
0810 06:49:29.793998 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.794004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.794092 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.794235 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.795131 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.796287 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.798066 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.798809 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.798945 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.802374 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.802381 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.802386 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.802424 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.805627 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.806389 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.806395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.806399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.806538 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.806941 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.806945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.806949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.807367 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.807524 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.807974 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.808204 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:49:29.814291 16961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895: Bootstrap starting.
I20260810 06:49:29.818128 16961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:29.819418 16961 log.cc:824] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:29.822510 16961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895: No bootstrap required, opened a new log
I20260810 06:49:29.827558 16961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } }
I20260810 06:49:29.827852 16961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffa11886a4f4c64b130a9fd2df89895, State: Initialized, Role: FOLLOWER
I20260810 06:49:29.828274 16961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } }
I20260810 06:49:29.829178 17228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:29.829545 17228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:29.830317 16961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:29.830451 16924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:49:29.830654 16924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:29.834262 16924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } }
I20260810 06:49:29.837342 17158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9ffa11886a4f4c64b130a9fd2df89895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75"
I20260810 06:49:29.837831 17158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:29.841311 17158 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ffa11886a4f4c64b130a9fd2df89895 in term 1.
I20260810 06:49:29.841990 16884 leader_election.cc:302] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e439a4a9b94cfcbfc1f2bbaee56c75, 9ffa11886a4f4c64b130a9fd2df89895; no voters:
I20260810 06:49:29.842499 17228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:29.842525 16924 leader_election.cc:288] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [CANDIDATE]: Term 1 election: Requested vote from peers c16faafa8bba41868c48338647ff1ffd (127.10.242.61:38385), 25e439a4a9b94cfcbfc1f2bbaee56c75 (127.10.242.60:41367)
I20260810 06:49:29.850514 17228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [term 1 LEADER]: Becoming Leader. State: Replica: 9ffa11886a4f4c64b130a9fd2df89895, State: Running, Role: LEADER
I20260810 06:49:29.851112 17228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } }
I20260810 06:49:29.855108 17242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ffa11886a4f4c64b130a9fd2df89895. Latest consensus state: current_term: 1 leader_uuid: "9ffa11886a4f4c64b130a9fd2df89895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:29.855463 17242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:29.859858 16853 master_runner.cc:186] Error getting master registration for 127.10.242.61:38385: OK, instance_id { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" instance_seqno: 1786344569166950 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:49:29.872545 16853 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 06:49:29.876832 17248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:49:29.877039 17248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:49:29.877243 17250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:29.879310 17250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:29.889712 17087 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } attempt: 5
Aug 10 06:49:29 dist-test-slave-zj8q krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344569, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.242.61@KRBTEST.COM for kudu/127.10.242.60@KRBTEST.COM
W20260810 06:49:29.903661 16891 tablet.cc:2369] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:29.906944 17158 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [term 1 FOLLOWER]: Refusing update from remote peer 9ffa11886a4f4c64b130a9fd2df89895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:29.908030 17242 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:49:29.911448 17126 tablet.cc:2369] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:29.914319 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.890655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.242.60:41367 (local address 127.0.0.1:35402)
0810 06:49:29.890905 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:29.890923 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:29.891021 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:29.891701 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:29.891706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:29.892201 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:29.892399 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.892412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.895409 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.895414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.896882 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:29.896898 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.898637 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.898644 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:29.898888 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.899605 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:29.899630 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:29.902921 (+ 3291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:29.910649 (+ 7728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.910653 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.910657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.911188 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.912088 (+ 900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:29.912093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:29.912097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:29.912257 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:29.913315 (+ 1058us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:29.913320 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:29.913490 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 06:49:29.913624 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 06:49:29.913917 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260810 06:49:29.916530 17261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.890380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35402 (local address 127.10.242.60:41367)
0810 06:49:29.891181 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.891187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.891212 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:49:29.891315 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.891321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.891456 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.892595 (+ 1139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.893073 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.895187 (+ 2114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.897155 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.898407 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.903172 (+ 4765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.903537 (+ 365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.903573 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.903578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.903647 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.910480 (+ 6833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.911484 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.911515 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.911522 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.911892 (+ 370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.912443 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.912448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.912451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.913107 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.915094 (+ 1987us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.915363 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.915742 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":707,"threads_started":1}
I20260810 06:49:29.922580 17087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd: Bootstrap starting.
I20260810 06:49:29.924105 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.868298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.10.242.61:38385)
0810 06:49:29.868477 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.868483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.885926 (+ 17443us) server_negotiation.cc:407] Connection header received
0810 06:49:29.886997 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.887002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.887089 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.887163 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.894583 (+ 7420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.895709 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.900732 (+ 5023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.901418 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.901511 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.915708 (+ 14197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.915715 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.915720 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.915755 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.919159 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.920771 (+ 1612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.920777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.920780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.920931 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.922337 (+ 1406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.922342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.922345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.922725 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.922962 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.923555 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.923786 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:49:29.926558 17087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:29.927858 17087 log.cc:824] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:29.929071 17201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.870031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.10.242.60:41367)
0810 06:49:29.870215 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.870221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.886725 (+ 16504us) server_negotiation.cc:407] Connection header received
0810 06:49:29.889452 (+ 2727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.889458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.889539 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.889604 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.897545 (+ 7941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.898720 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.905292 (+ 6572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.906074 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.906201 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.920021 (+ 13820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.920028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.920032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.920068 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.923762 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.924989 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.924995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.924999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.925146 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.926886 (+ 1740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.926891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.926895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.927220 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.927437 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.928479 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.928733 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:49:29.937063 17087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd: No bootstrap required, opened a new log
I20260810 06:49:29.937552 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:29.883766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55554 (local address 127.10.242.62:43197)
0810 06:49:29.883931 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:29.883938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:29.892027 (+ 8089us) server_negotiation.cc:407] Connection header received
0810 06:49:29.892250 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:29.892256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:29.892331 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:29.892396 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:29.902320 (+ 9924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.903439 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.910265 (+ 6826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:29.910918 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:29.911014 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:29.926970 (+ 15956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:29.926977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:29.926982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:29.927024 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:29.933085 (+ 6061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.934335 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.934340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.934344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.934523 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:29.935273 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:29.935279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:29.935282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:29.935653 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:29.935912 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:29.936776 (+ 864us) server_negotiation.cc:299] Negotiation successful
0810 06:49:29.937053 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:49:29.951962 17208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ffa11886a4f4c64b130a9fd2df89895. Latest consensus state: current_term: 1 leader_uuid: "9ffa11886a4f4c64b130a9fd2df89895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:29.952291 17208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:29.955587 17087 raft_consensus.cc:348] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } }
I20260810 06:49:29.955981 17087 raft_consensus.cc:729] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c16faafa8bba41868c48338647ff1ffd, State: Initialized, Role: FOLLOWER
I20260810 06:49:29.956487 17087 consensus_queue.cc:260] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } }
I20260810 06:49:29.970450 17228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ffa11886a4f4c64b130a9fd2df89895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:29.970783 17228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:29.975170 17250 catalog_manager.cc:1347] Generated new cluster ID: 5e2c973e31fa48a0b7e60db3cc46e914
I20260810 06:49:29.975273 17250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:29.989579 17087 sys_catalog.cc:564] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:29.990103 17048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:29.989647 17049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9ffa11886a4f4c64b130a9fd2df89895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c16faafa8bba41868c48338647ff1ffd"
I20260810 06:49:29.994438 17048 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [term 1 FOLLOWER]: Refusing update from remote peer 9ffa11886a4f4c64b130a9fd2df89895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:29.995342 17242 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:29.998142 17049 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ffa11886a4f4c64b130a9fd2df89895 in term 1.
I20260810 06:49:29.988830 17267 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:30.010288 17267 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:30.019433 17208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ffa11886a4f4c64b130a9fd2df89895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:30.021006 17208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:30.041677 17274 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ffa11886a4f4c64b130a9fd2df89895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:30.041836 17267 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ffa11886a4f4c64b130a9fd2df89895. Latest consensus state: current_term: 1 leader_uuid: "9ffa11886a4f4c64b130a9fd2df89895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:30.042100 17274 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:30.042128 17267 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:49:30.043689 17228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ffa11886a4f4c64b130a9fd2df89895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffa11886a4f4c64b130a9fd2df89895" member_type: VOTER last_known_addr { host: "127.10.242.62" port: 43197 } } peers { permanent_uuid: "c16faafa8bba41868c48338647ff1ffd" member_type: VOTER last_known_addr { host: "127.10.242.61" port: 38385 } } peers { permanent_uuid: "25e439a4a9b94cfcbfc1f2bbaee56c75" member_type: VOTER last_known_addr { host: "127.10.242.60" port: 41367 } } }
I20260810 06:49:30.043980 17228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:30.046485 17250 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:30.048712 17250 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:30.063221 17250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ffa11886a4f4c64b130a9fd2df89895: Generated new TSK 0
I20260810 06:49:30.064253 17250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:49:30.106393 17293 catalog_manager.cc:1259] Loaded cluster ID: 5e2c973e31fa48a0b7e60db3cc46e914
I20260810 06:49:30.106616 17293 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd: loading cluster ID for follower catalog manager: success
I20260810 06:49:30.110385 17293 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd: acquiring CA information for follower catalog manager: success
I20260810 06:49:30.112874 17293 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c16faafa8bba41868c48338647ff1ffd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 06:49:30.211175 11208 mini_kdc.cc:348] Time spent kdestroy: real 0.129s user 0.005s sys 0.024s
W20260810 06:49:30.270272 17092 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.259362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.10.242.61:38385)
0810 06:49:30.259601 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.259608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.268121 (+ 8513us) server_negotiation.cc:407] Connection header received
0810 06:49:30.268632 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.268638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.268730 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.268809 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.269989 (+ 1180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60044: BlockingRecv error: recv got EOF from 127.0.0.1:60044 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 06:49:30.272852 16968 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.259130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.10.242.62:43197)
0810 06:49:30.259339 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.259346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.271246 (+ 11900us) server_negotiation.cc:407] Connection header received
0810 06:49:30.271505 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.271511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.271589 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.271652 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.272628 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55558: BlockingRecv error: recv got EOF from 127.0.0.1:55558 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 06:49:30.280500 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.274619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35408 (local address 127.10.242.60:41367)
0810 06:49:30.274803 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.274810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.278714 (+ 3904us) server_negotiation.cc:407] Connection header received
0810 06:49:30.278966 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.278971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.279061 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.279164 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.280213 (+ 1049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35408: BlockingRecv error: recv got EOF from 127.0.0.1:35408 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:49:30.324095 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.292417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60054 (local address 127.10.242.61:38385)
0810 06:49:30.292595 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.292601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.305719 (+ 13118us) server_negotiation.cc:407] Connection header received
0810 06:49:30.306108 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.306115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.306372 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.306512 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.307245 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.308429 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.310433 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.311108 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.311219 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.322468 (+ 11249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.323257 (+ 789us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.323378 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.323649 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.323756 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:49:30.375033 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.368098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.10.242.60:41367)
0810 06:49:30.368324 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.368331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.373071 (+ 4740us) server_negotiation.cc:407] Connection header received
0810 06:49:30.373431 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.373439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.373571 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.373653 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.374807 (+ 1154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35412: BlockingRecv error: recv got EOF from 127.0.0.1:35412 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:49:30.409459 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.391191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60062 (local address 127.10.242.61:38385)
0810 06:49:30.391395 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.391401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.402721 (+ 11320us) server_negotiation.cc:407] Connection header received
0810 06:49:30.402956 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.402963 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.403057 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.403134 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.403849 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.405022 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.407062 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.407783 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.407884 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.408535 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.408707 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.408775 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.409017 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.409114 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:49:30.432482 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.424059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35424 (local address 127.10.242.60:41367)
0810 06:49:30.424247 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.424253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.430637 (+ 6384us) server_negotiation.cc:407] Connection header received
0810 06:49:30.431007 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.431013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.431122 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.431194 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.432244 (+ 1050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35424: BlockingRecv error: recv got EOF from 127.0.0.1:35424 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:49:30.452301 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.445040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.10.242.61:38385)
0810 06:49:30.445217 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.445223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.445713 (+ 490us) server_negotiation.cc:407] Connection header received
0810 06:49:30.446024 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.446031 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.446122 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.446208 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.446891 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.448012 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.450002 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.450660 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.450759 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.451382 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.451583 (+ 201us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.451649 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.451873 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.451974 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 06:49:30.462728 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.459824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35426 (local address 127.10.242.60:41367)
0810 06:49:30.460011 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.460017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.461063 (+ 1046us) server_negotiation.cc:407] Connection header received
0810 06:49:30.461340 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.461346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.461441 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.461507 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.462495 (+ 988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35426: BlockingRecv error: recv got EOF from 127.0.0.1:35426 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:49:30.486380 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.481168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60092 (local address 127.10.242.61:38385)
0810 06:49:30.481450 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.481456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.481613 (+ 157us) server_negotiation.cc:407] Connection header received
0810 06:49:30.481830 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.481836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.481911 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.481995 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.482454 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.483268 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.484616 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.485109 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.485237 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.485633 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.485751 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.485830 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.485969 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.486058 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
W20260810 06:49:30.497680 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.495629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35440 (local address 127.10.242.60:41367)
0810 06:49:30.495816 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.495821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.496260 (+ 439us) server_negotiation.cc:407] Connection header received
0810 06:49:30.496448 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.496454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.496543 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.496628 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.497436 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35440: BlockingRecv error: recv got EOF from 127.0.0.1:35440 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:49:30.513985 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.508555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.10.242.61:38385)
0810 06:49:30.508731 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.508737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.509011 (+ 274us) server_negotiation.cc:407] Connection header received
0810 06:49:30.509167 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.509172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.509256 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.509357 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.509706 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.510808 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.512082 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.512763 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.512897 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.513244 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.513392 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.513519 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.513594 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.513674 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 06:49:30.521505 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.519556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.10.242.60:41367)
0810 06:49:30.519777 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.519783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.520136 (+ 353us) server_negotiation.cc:407] Connection header received
0810 06:49:30.520325 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.520329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.520399 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.520475 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.521246 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35452: BlockingRecv error: recv got EOF from 127.0.0.1:35452 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:49:30.546653 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.541021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.10.242.61:38385)
0810 06:49:30.541260 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.541271 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.541666 (+ 395us) server_negotiation.cc:407] Connection header received
0810 06:49:30.541883 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.541887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.541967 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.542067 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.542587 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.543385 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.544773 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.545302 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.545431 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.545868 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.546000 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.546079 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.546293 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.546383 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 06:49:30.557245 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.555355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.10.242.60:41367)
0810 06:49:30.555518 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.555525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.555890 (+ 365us) server_negotiation.cc:407] Connection header received
0810 06:49:30.556044 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.556048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.556122 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.556204 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.557017 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35462: BlockingRecv error: recv got EOF from 127.0.0.1:35462 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:49:30.599491 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.593708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60118 (local address 127.10.242.61:38385)
0810 06:49:30.593914 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.593921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.594283 (+ 362us) server_negotiation.cc:407] Connection header received
0810 06:49:30.594483 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.594489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.594579 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.594694 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.595043 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.596192 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.597565 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.598271 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.598396 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.598762 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.598898 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.598994 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.599077 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.599163 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 06:49:30.609884 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.607987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.10.242.60:41367)
0810 06:49:30.608211 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.608217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.608534 (+ 317us) server_negotiation.cc:407] Connection header received
0810 06:49:30.608707 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.608711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.608783 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.608859 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.609668 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35468: BlockingRecv error: recv got EOF from 127.0.0.1:35468 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:49:30.685076 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.679535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60128 (local address 127.10.242.61:38385)
0810 06:49:30.679771 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.679778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.680360 (+ 582us) server_negotiation.cc:407] Connection header received
0810 06:49:30.680575 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.680579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.680660 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.680742 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.681222 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.681978 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.683357 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.683793 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.683913 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.684356 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.684469 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.684547 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.684708 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.684795 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 06:49:30.692402 17201 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:49:30.690439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35470 (local address 127.10.242.60:41367)
0810 06:49:30.690657 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.690663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.690984 (+ 321us) server_negotiation.cc:407] Connection header received
0810 06:49:30.691217 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.691221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.691291 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.691363 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:30.692171 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35470: BlockingRecv error: recv got EOF from 127.0.0.1:35470 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:49:30.767907 17222 catalog_manager.cc:1259] Loaded cluster ID: 5e2c973e31fa48a0b7e60db3cc46e914
I20260810 06:49:30.768014 17222 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75: loading cluster ID for follower catalog manager: success
I20260810 06:49:30.770570 17222 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75: acquiring CA information for follower catalog manager: success
I20260810 06:49:30.772282 17222 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 25e439a4a9b94cfcbfc1f2bbaee56c75: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:49:30.828588 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.824092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.10.242.61:38385)
0810 06:49:30.824348 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.824354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.824579 (+ 225us) server_negotiation.cc:407] Connection header received
0810 06:49:30.824755 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.824760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.824833 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.824912 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.825220 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.825911 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.827156 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.827576 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.827675 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.827987 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.828102 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.828166 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.828259 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.828323 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 06:49:30.839781 17201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.834210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35480 (local address 127.10.242.60:41367)
0810 06:49:30.834411 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.834418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.834759 (+ 341us) server_negotiation.cc:407] Connection header received
0810 06:49:30.834947 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.834951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.835104 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.835215 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.835603 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.836324 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.837601 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.838027 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.838165 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.838643 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.839218 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.839353 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.839454 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.839535 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:49:30.852355 17201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.845338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35482 (local address 127.10.242.60:41367)
0810 06:49:30.845508 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.845513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.846844 (+ 1331us) server_negotiation.cc:407] Connection header received
0810 06:49:30.846982 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.846987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.847072 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.847164 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.847960 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.848952 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.850741 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.851202 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.851305 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.851704 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.851794 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.851862 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.852000 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.852074 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:49:30.853372 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.845044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55560 (local address 127.10.242.62:43197)
0810 06:49:30.845692 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.845698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.847459 (+ 1761us) server_negotiation.cc:407] Connection header received
0810 06:49:30.847679 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.847685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.847911 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.848078 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.848523 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.849336 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.850943 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.851569 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.851688 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.852331 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.852746 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.852852 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.852978 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.853068 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":484,"threads_started":1}
I20260810 06:49:30.854063 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:30.845173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.10.242.61:38385)
0810 06:49:30.845350 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:30.845355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:30.846632 (+ 1277us) server_negotiation.cc:407] Connection header received
0810 06:49:30.846925 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:30.846931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:30.847020 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:30.847083 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:30.847865 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.848945 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.852046 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:30.852669 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:30.852762 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:30.853396 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:30.853522 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:30.853616 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:30.853739 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 06:49:30.853820 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:49:30.857581 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16853
I20260810 06:49:30.858598 16953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:30.971674 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16853
I20260810 06:49:31.001072 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 16980
I20260810 06:49:31.002271 17079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:31.108740 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 16980
I20260810 06:49:31.134393 11208 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu pid 17095
I20260810 06:49:31.135524 17188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:31.240350 11208 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcYuaB1/build/asan/bin/kudu with pid 17095
2026-08-10T06:49:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4544 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9552 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (364722 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST